Check for security.txt files under the top-level directory too.

[EdOverflow]

Hi @Pinjasaur,

First off, fantastic work! I enjoyed your write-up and thank you for sharing your tool. I just wanted to point out that you are currently only checking for security.txt files under the .well-known path, but it is worth noting that the Internet Draft also allows security.txt files to be served under the top-level directory (https://example.com/security.txt). You might stumble across a couple of hosts only hosting their security.txt file under the top-level directory; e.g. https://bit.ly/security.txt.

tfc/tfc.js

Line 64 in fa2fb44

const files = ['robots.txt', 'humans.txt', '.well-known/security.txt']

Keep up the excellent work. :)

[Pinjasaur]

Hi @EdOverflow!

Thanks for your work on security.txt! This wouldn't be possible without your work.

I see that v06 of the Internet Draft was released a few days ago: https://tools.ietf.org/html/draft-foudil-securitytxt-06

When I was building this I was referring to v05 released back in January. I see that v06 did answer one of my questions from then:

... MUST be served with "https" (as per section 2.7.2 of [RFC7230]) ...

I'll create a new issue for only requesting security.txt files using https:// since that's separate from what you reported, but needs to be updated.

---

Now, onto your comment:

I do remember finding this when I was initially researching & writing this. Looks like v06 changed the wording a bit from v05 (and removed the ASCII art diagram). Hoping you can clarify:

• The security.txt file should exist under /.well-known/:
• If it does not, check the / (root) path? The root path should redirect to /.well-known/?

It does not seem like I can just check the root path, since not all hosts redirect to /.well-known/. Just now, I checked Google and the file only exists under the /.well-known/ path.

I'd like to get this updated in order to get the most accurate results, but I want to make sure I fully understand the implications and best way to handle attempting files (what directory, what order, etc.).

Thanks again!

[EdOverflow]

If it does not, check the / (root) path? The root path should redirect to /.well-known/?

We recommend people use /.well-known/ instead of the top-level directory, but the specification still allows you to set up a security.txt file under the root directory — and not use /.well-known/ at all. The top-level path alternative was introduced to ensure that even website owners who cannot use the /.well-known/ path, can serve a security.txt file.

You will see cases where a website only serves a security.txt file under the root directory. For this reason, projects such as https://securitytext.org/ check for both /.well-known/security.txt and /security.txt.

I hope that clears up any uncertainties. :)

[Pinjasaur]

Thanks @EdOverflow. I'll look into how to implement this properly.