Turn what outside AI pen testers proved into checks that cost nothing to run

Five vendors tested a PHP application.  The weaknesses they proved now have
code rules: Dompdf running PHP inside rendered documents, the request path
written into a page unescaped, a formula guard that only looks at the first
character, uploads accepted on the browser's content type, address checks
that miss the 100.64.0.0/10 shared range, and forwarding headers trusted
from anywhere.  Each was run against about twelve thousand files of other
code and reported only real problems.

New web checks find a dependency lock file served from the web root and
extra path after a PHP script written into the page, both confirmed against
real Apache and PHP.  The open redirect check adds the path form and stops
reporting a redirect to the site's own sign in page, using a new
header_matches test in the rule language.  Code reasoning names the access
mistakes those testers found most, without asking again about code it has
already read.

Author: hackrange

app/core/code_reasoning.py

@@ -164,16 +164,26 @@ def mask(text: str) -> str:
 Report only these, and only when the code you were given shows it:
   object-authz    an id or key from the request reaches a read, change or delete \
 of a stored object, and nothing checks that the caller owns it, is assigned to \
-it, or may otherwise reach it.  A role check alone is not enough.
+it, or may otherwise reach it.  A role check alone is not enough.  This \
+includes a child id (a question, answer, note, line item or file) never checked \
+to belong to the parent object or public token the caller holds, and a search, \
+list, autocomplete or export that returns objects the same caller is refused \
+when asking for one by id.
   function-authz  a privileged or state changing action that a lower role can \
 reach, because the server checks only that somebody is signed in, or checks \
-the wrong role.  Hiding a button is not a check.
+the wrong role.  Hiding a button is not a check.  This includes an action \
+that spends money or quota (a model call, a mass update, an export of \
+everything) open to a read only role.
   field-authz     individual fields or questions with their own edit or view \
-rules are written or returned without applying those rules.
+rules are written or returned without applying those rules, or an owner, \
+assignee or user id field set from the request to a user the caller could not \
+otherwise pick.
   csrf            a browser reachable state change whose forgery check is \
 missing or does nothing.
   auth-logic      sign in, second factor, session or single sign on logic that \
-can be bypassed, replayed, or have its lockout reset.
+can be bypassed, replayed, or have its lockout reset; a session or token \
+that keeps working after its user is deactivated or loses a role; a one time \
+code or public link that still works after the record it opened is finished.
   trust-client    a value only the server should decide (a price, a score, a \
 role, a verdict) taken from the request as is.
   model-trust     a language model's answer is used to lower, close, waive, \

app/core/dast_checks/curated/business-logic-open-redirect.json

@@ -14,7 +14,9 @@
     "/?returnUrl=https://canary-openredirect.example/",
     "/?dest=https://canary-openredirect.example/",
     "/login?next=https://canary-openredirect.example/",
-    "/logout?next=https://canary-openredirect.example/"
+    "/logout?next=https://canary-openredirect.example/",
+    "/%5Ccanary-openredirect.example/%2f..",
+    "/%2F%2Fcanary-openredirect.example/%2f.."
   ],
   "probe": [
     {
@@ -27,10 +29,10 @@
   ],
   "confirm": {
     "control": "A path that cannot exist, so the site's not found answer is known and a real redirect to the foreign host is told apart from it.",
-    "differential": "status_in(301, 302, 303, 307, 308) and \"canary-openredirect.example\" in header(\"location\")",
+    "differential": "status_in(301, 302, 303, 307, 308) and header_matches(\"location\", \"^\\\\s*(https?:)?[/\\\\\\\\]{2}canary-openredirect\\\\.example\")",
     "retest": 2
   },
-  "reproduce": "curl -sk -i 'https://host/?next=https://canary-openredirect.example/' and check the response is a 30x whose Location header points at canary-openredirect.example. Try the other parameter names (url, redirect, return, returnUrl, dest) and the login and logout paths. A redirect to an outside host means the parameter is an open redirect.",
+  "reproduce": "curl -sk -i 'https://host/?next=https://canary-openredirect.example/' and check the response is a 30x whose Location header points at canary-openredirect.example. Try the other parameter names (url, redirect, return, returnUrl, dest) and the login and logout paths. A path form also works on some servers: curl -sk -i 'https://host/%5Ccanary-openredirect.example/%2f..' answering with Location: /\\canary-openredirect.example/..., which a browser reads as the outside host. A redirect to an outside host means the parameter is an open redirect.",
   "remediate": "Do not redirect to a location taken from the request. Allow only relative paths, or an explicit allow list of hosts, and reject or ignore an absolute URL to another site.",
   "references": [],
   "mappings": {

app/core/dast_checks/curated/injection-php-path-info-reflected.json

@@ -0,0 +1,45 @@
+{
+  "id": "injection-php-path-info-reflected",
+  "class": "injection",
+  "safety": "read",
+  "severity_floor": "medium",
+  "title": "Extra path after a PHP script is written into the page unescaped",
+  "summary": "A PHP page accepts extra path after its own name (page.php/anything) and writes it back into the HTML without encoding it, usually through $_SERVER['PHP_SELF'] in a form action or a link. A link with markup in that extra path puts the markup on the page: a meta refresh that sends the visitor to any site, a fake sign in form, or script where the content security policy allows it.",
+  "needs": [],
+  "candidate_paths": [
+    "/index.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/login.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/admin.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/search.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/contact.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/register.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/profile.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/dashboard.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/forgot-password.php/gcrpi%22%3E%3Cgcrpi7q%3E",
+    "/reset-password.php/gcrpi%22%3E%3Cgcrpi7q%3E"
+  ],
+  "probe": [
+    {
+      "request": {
+        "method": "GET",
+        "path": "{route}",
+        "follow": false
+      }
+    }
+  ],
+  "confirm": {
+    "control": "A path that cannot exist.  The marker only ever goes in the probe, so a page that shows it unescaped wrote the request's own path into its markup.",
+    "differential": "status_in(200) and body_contains('\"><gcrpi7q>') and 'html' in header('content-type')",
+    "retest": 2
+  },
+  "reproduce": "curl -sk \"https://host/index.php/x%22%3E%3Cb%3Etest%3C/b%3E\" and look for the literal text \"><b>test</b> in the page, unescaped (not &quot;&gt;&lt;b&gt;). Try the other script names the rule lists, and any other PHP page on the site. In a browser, a meta refresh in the same place sends the visitor to another site.",
+  "remediate": "Encode the value before writing it into HTML: htmlspecialchars($_SERVER['PHP_SELF'], ENT_QUOTES, 'UTF-8'). Better, do not use PHP_SELF or REQUEST_URI in markup at all: leave a form's action empty to post back to the same page, or write the script's fixed path. Turning off AcceptPathInfo in Apache also stops the extra path from reaching the script.",
+  "references": [
+    "https://owasp.org/www-community/attacks/xss/"
+  ],
+  "mappings": {
+    "cwe": "CWE-79",
+    "owasp": "A03:2021"
+  },
+  "rulesets": []
+}

app/core/dast_checks/curated/misconfiguration-dependency-lockfile-served.json

@@ -0,0 +1,43 @@
+{
+  "id": "misconfiguration-dependency-lockfile-served",
+  "class": "misconfiguration",
+  "safety": "read",
+  "severity_floor": "low",
+  "anonymous": true,
+  "title": "Dependency manifest or lock file served from the web root",
+  "summary": "The site serves its composer, npm, yarn, Ruby or Python lock file to anybody. It lists every library and its exact version, which tells an attacker which known vulnerabilities to try first, and it shows that the project folder itself is the web root, so other files beside it may be served too.",
+  "needs": [],
+  "candidate_paths": [
+    "/composer.lock",
+    "/composer.json",
+    "/package-lock.json",
+    "/yarn.lock",
+    "/Gemfile.lock",
+    "/Pipfile.lock",
+    "/poetry.lock"
+  ],
+  "probe": [
+    {
+      "request": {
+        "method": "GET",
+        "path": "{route}",
+        "follow": false
+      }
+    }
+  ],
+  "confirm": {
+    "control": "A path that cannot exist, so a site that answers every path with the same page is not mistaken for one serving the file.",
+    "differential": "status_in(200) and body_differs(control) and body_matches(\"(?m)\\\"content-hash\\\"\\\\s*:\\\\s*\\\"[0-9a-f]{32}\\\"|\\\"lockfileVersion\\\"\\\\s*:\\\\s*\\\\d|^# yarn lockfile v1|^GEM\\\\s*$\\\\s+remote:|\\\"_meta\\\"\\\\s*:\\\\s*\\\\{\\\\s*\\\"hash\\\"|^# This file is automatically @generated by Poetry|\\\"require\\\"\\\\s*:\\\\s*\\\\{[^}]*\\\"php\\\"\\\\s*:\")",
+    "retest": 2
+  },
+  "reproduce": "curl -sk 'https://host/composer.lock' | head -20 and look for \"content-hash\" and a \"packages\" list. Try package-lock.json (\"lockfileVersion\"), yarn.lock (\"# yarn lockfile v1\"), Gemfile.lock (\"GEM\" then \"remote:\"), Pipfile.lock (\"_meta\") and poetry.lock. Compare with a path that cannot exist, so a page served for every path is not mistaken for the file.",
+  "remediate": "Serve the application from a public folder that holds only what browsers need, not from the project root. If that cannot change yet, deny dependency and lock files in the web server (for Apache: <FilesMatch \"^(composer\\.(json|lock)|package(-lock)?\\.json|yarn\\.lock|Gemfile\\.lock|Pipfile\\.lock|poetry\\.lock)$\"> Require all denied </FilesMatch>).",
+  "references": [
+    "https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/04-Review_Old_Backup_and_Unreferenced_Files_for_Sensitive_Information"
+  ],
+  "mappings": {
+    "cwe": "CWE-538",
+    "owasp": "A05:2021"
+  },
+  "rulesets": []
+}

app/core/dast_checks/engine.py

@@ -112,7 +112,7 @@ def request(self, method: str, path: str, query: dict = None,
 
 _ALLOWED_FUNCS = {"body_differs", "body_contains", "header", "reflected",
                   "status_in", "differ", "same", "status_of",
-                  "body_matches", "cookie_matches"}
+                  "body_matches", "cookie_matches", "header_matches"}
 _ALLOWED_CMP = (ast.Eq, ast.NotEq, ast.Lt, ast.LtE, ast.Gt, ast.GtE,
                 ast.In, ast.NotIn)
 
@@ -264,6 +264,10 @@ def status_of(resp):
     def body_matches(pattern):
         return bool(_rule_regex(pattern).search(body[:MATCH_BODY_CHARS]))
 
+    def header_matches(name, pattern):
+        """True when the named response header matches the pattern."""
+        return bool(_rule_regex(pattern).search(headers.get(str(name).lower(), "")[:4000]))
+
     def cookie_matches(pattern):
         """True when any cookie the response set matches, attributes included."""
         rx = _rule_regex(pattern)
@@ -279,7 +283,7 @@ def cookie_matches(pattern):
                    "header": header, "reflected": reflected,
                    "status_in": status_in, "differ": differ, "same": same,
                    "status_of": status_of, "body_matches": body_matches,
-                   "cookie_matches": cookie_matches},
+                   "cookie_matches": cookie_matches, "header_matches": header_matches},
     }
     env.update(saved or {})
     return env