Explain each weakness as a grounded risk: what can go wrong, how it is attacked, how likely

The enhanced review now writes each finding as a short risk narrative, not
one line: what an attacker gains, how they would attack this finding at this
address, and how likely it is to lead to a breach.  The likelihood reasons
from the site's exposure (reachable from the open internet raises it, allow
listed lowers it), the attack surface and reachability, any compensating
control visible in the evidence, and the reality that attackers now automate
discovery and exploitation at scale, so an internet reachable, easily
automated weakness is reached sooner than its manual difficulty suggests.

Real world examples are kept factual.  They do not come from the model, which
would invent them.  A vetted table maps each weakness class to a documented
public incident and a checkable reference, the report renders that straight
from the table, and the model is handed the matching entry as a fact to reason
from with a hard rule never to invent a case, a number or an identifier.  The
review is also given the deterministic risk score and the exposure so its words
match the numbers.  A finding with no documented precedent gets none rather
than an inflated one.

The site page and the PDF show how a weakness class is abused and its real
world precedent, from the vetted table, even before the enhanced review runs.

Author: hackrange

app/core/dast_precedent.py

@@ -0,0 +1,167 @@
+"""Verified real world precedent for a weakness class.
+
+The point is factual grounding.  A language model asked for a real world
+example of a weakness will happily invent a CVE number, a company and a breach
+size.  So the examples do not come from the model: they live here, vetted, each
+one a well documented public incident with a reference a reader can check.  The
+triage prompt is handed the matching entry as a fact to reason from, and the
+report renders it straight from this table, so what a reader sees is sourced
+here, not from the model's memory.
+
+Each entry is one weakness class: how an attacker abuses it, a documented real
+world case, and a checkable reference.  A class is only listed when there is a
+genuinely well documented public incident for it; a low or informational
+finding with no real precedent gets none, rather than an inflated one.
+
+Match order matters: the most specific, highest impact classes come first, so
+an injection that leads to code execution is read as code execution, not as a
+generic injection.
+"""
+
+from __future__ import annotations
+
+import re
+
+# Each: a matcher over the finding's text, the OWASP and CWE anchors, how the
+# class is abused, and a documented real world case with a reference.  Keep the
+# facts conservative and checkable; never add a case that is not well sourced.
+PRECEDENTS = [
+    {
+        "key": "rce",
+        "match": re.compile(r"remote code|command inj|\brce\b|code exec|os command|"
+                            r"deserial|CWE-?(?:78|94|77|502)\b", re.I),
+        "ref": "CVE-2017-5638; OWASP A03:2021",
+        "abuse": "The attacker sends crafted input that the server runs as code, "
+                 "takes control of the host, then reaches the data and systems "
+                 "behind it.",
+        "precedent": "The 2017 Equifax breach, which exposed data on about 147 "
+                     "million people, began with an unpatched remote code "
+                     "execution flaw in Apache Struts (CVE-2017-5638).",
+    },
+    {
+        "key": "ssrf",
+        "match": re.compile(r"\bssrf\b|server.side request|CWE-?918\b", re.I),
+        "ref": "OWASP A10:2021; CWE-918",
+        "abuse": "The attacker makes the server fetch a URL of their choosing, "
+                 "reaching internal services the outside cannot, such as a cloud "
+                 "metadata endpoint, and reading back what it returns.",
+        "precedent": "The 2019 Capital One breach, which affected about 100 "
+                     "million people, used server side request forgery to reach "
+                     "the cloud metadata service and take credentials from it.",
+    },
+    {
+        "key": "sqli",
+        "match": re.compile(r"sql\s*inj|\bsqli\b|CWE-?89\b", re.I),
+        "ref": "OWASP A03:2021; CWE-89",
+        "abuse": "The attacker puts database syntax into a parameter to read or "
+                 "change data the query never meant to expose, often dumping whole "
+                 "tables of accounts or payment records.",
+        "precedent": "SQL injection was the entry point for the 2008 Heartland "
+                     "Payment Systems breach, about 130 million card numbers, and "
+                     "the 2015 TalkTalk breach, for which the UK regulator fined "
+                     "the company.",
+    },
+    {
+        "key": "xss",
+        "match": re.compile(r"\bxss\b|cross.site.script|CWE-?79\b", re.I),
+        "ref": "OWASP A03:2021; CWE-79",
+        "abuse": "The attacker gets script to run in another user's session, so it "
+                 "acts as that user: stealing the session, changing their account, "
+                 "or spreading to everyone who views the page.",
+        "precedent": "The 2005 Samy worm used a stored cross site script to add "
+                     "over a million MySpace friends in under a day, by running in "
+                     "the session of everyone who viewed an infected profile.",
+    },
+    {
+        "key": "bola",
+        "match": re.compile(r"\bidor\b|\bbola\b|broken object|object level auth|"
+                            r"insecure direct object|CWE-?639\b", re.I),
+        "ref": "OWASP API1:2023; CWE-639",
+        "abuse": "The attacker changes an id in a URL or request and reads or edits "
+                 "records that belong to other users, walking the ids to reach many "
+                 "accounts at once.",
+        "precedent": "The 2019 First American Financial exposure, about 885 million "
+                     "documents, and the 2018 USPS Informed Visibility API both let "
+                     "anyone read other people's records by changing an id, each "
+                     "documented publicly at the time.",
+    },
+    {
+        "key": "secret",
+        "match": re.compile(r"api.?key|secret|hardcoded|private key|credential|"
+                            r"access.key|\.htpasswd|\.npmrc|id_rsa|CWE-?798\b", re.I),
+        "ref": "CWE-798; OWASP A05:2021",
+        "abuse": "The attacker reads the exposed key or password and uses it to sign "
+                 "in to the service it belongs to, with whatever access that key "
+                 "was granted, no guessing required.",
+        "precedent": "The 2016 Uber breach, about 57 million riders and drivers, "
+                     "happened because a cloud access key sat in a code repository "
+                     "the attackers were able to reach.",
+    },
+    {
+        "key": "graphql-amplification",
+        "match": re.compile(r"amplif|aliased|batch|rate limit|CWE-?770\b|"
+                            r"API4:2023", re.I),
+        "ref": "OWASP API4:2023; CWE-770",
+        "abuse": "The attacker packs many operations into one request, so a single "
+                 "call becomes many login or one time code attempts, and the per "
+                 "request rate limit and edge counting never see the real volume.",
+        "precedent": "Bypassing login and one time code rate limits by batching or "
+                     "aliasing GraphQL operations is a documented technique, "
+                     "recognised in the OWASP API Security Top 10 as unrestricted "
+                     "resource consumption.",
+    },
+    {
+        "key": "credential-stuffing",
+        "match": re.compile(r"brute.?forc|credential stuff|weak password|"
+                            r"password spray|CWE-?307\b", re.I),
+        "ref": "OWASP A07:2021; CWE-307",
+        "abuse": "The attacker tries large lists of guessed or reused passwords "
+                 "against the login, and every account whose password is weak or "
+                 "reused is taken over.",
+        "precedent": "Credential stuffing against Nintendo in 2020 compromised "
+                     "about 300,000 accounts by reusing passwords leaked from other "
+                     "sites.",
+    },
+    {
+        "key": "data-exposure",
+        "match": re.compile(r"directory listing|directory index|exposed backup|"
+                            r"\.bak\b|\.sql\b|open to the world|publicly readable|"
+                            r"exposed .*(database|dump|backup)|CWE-?(?:538|548)\b", re.I),
+        "ref": "OWASP A05:2021; CWE-200",
+        "abuse": "The attacker reads files or listings the site left open, taking "
+                 "backups, database dumps or source that were never meant to be "
+                 "public.",
+        "precedent": "Misconfigured, publicly readable cloud storage has exposed "
+                     "data repeatedly, such as the 2017 exposure of about 14 "
+                     "million Verizon customer records left open on a storage "
+                     "bucket.",
+    },
+]
+
+
+def match(f: dict):
+    """The precedent for a finding's weakness class, or None.
+
+    Matched on the finding's own text, most specific class first, so nothing is
+    attached unless the finding clearly belongs to a class with a documented
+    real world case."""
+    text = " ".join(str(f.get(k) or "") for k in ("title", "cwe", "evidence", "url"))
+    for p in PRECEDENTS:
+        if p["match"].search(text):
+            return {"class": p["key"], "abuse": p["abuse"],
+                    "precedent": p["precedent"], "reference": p["ref"]}
+    return None
+
+
+def annotate(findings: list) -> list:
+    """Add the verified abuse description, real world precedent and reference to
+    each finding that belongs to a documented class, in place.  Findings with no
+    documented precedent are left untouched, never given an inflated one."""
+    for f in findings:
+        p = match(f)
+        if p:
+            f["abuse"] = p["abuse"]
+            f["precedent"] = p["precedent"]
+            f["precedent_ref"] = p["reference"]
+            f["weakness_class"] = p["class"]
+    return findings

app/core/dast_report.py

@@ -344,9 +344,12 @@ def build(db, asset, scan, user=None, organization: str = "") -> dict:
         findings.append(f)
     overall = score(findings, triaged)
     # Risk, deterministic: likelihood x breachability, weighted by exposure.
-    from app.core import dast_risk
+    from app.core import dast_precedent, dast_risk
 
     dast_risk.annotate(findings, getattr(asset, "exposure", "") or "")
+    # The verified real world precedent for each weakness class, from the vetted
+    # table, so the report shows a factual comparable case even with no AI triage.
+    dast_precedent.annotate(findings)
     # Code sinks from the mapped repositories, split by whether the scan reached
     # the route they sit behind.  Reachable ones are leads to prove; the rest
     # are context to review, not dynamic bugs.

app/core/external.py

@@ -3738,26 +3738,52 @@ def recover() -> int:
 
 TRIAGE_SYSTEM = """You review the results of a dynamic application security scan \
 (DAST) of one public web site.  You also get the open static analysis (SAST) \
-findings from the source code repositories believed to be behind the site.
-
-For each DAST finding decide: likely_real, likely_false_positive, or \
-needs_review.  Scanners report many false positives: version banners that are \
-patched, generic header advice, reflected input that is encoded, and template \
-matches on a soft 404 page.  When a SAST finding in the code supports a DAST \
-finding (same weakness, same route or parameter), say so; that raises \
-confidence a great deal.
-
-A finding you call likely_false_positive is taken out of the results, so use \
-it only when the evidence shows it is noise.  When unsure, say needs_review.  \
-A finding about an exposed secret stays whatever you say, so judge it anyway.
-
-Do not change severities.  Do not invent findings.
+findings from the code behind the site, the site's exposure from a two vantage \
+check, and, for each finding, a deterministic risk score and, when the weakness \
+is a known class, a VERIFIED real world precedent.
+
+For each DAST finding do two things.
+
+1) Decide a verdict: likely_real, likely_false_positive, or needs_review.  \
+Scanners report many false positives: version banners that are patched, generic \
+header advice, reflected input that is encoded, and template matches on a soft \
+404 page.  A finding you call likely_false_positive is taken out of the \
+results, so use it only when the evidence shows it is noise.  When unsure, say \
+needs_review.  A finding about an exposed secret stays whatever you say.
+
+2) Write "why" as a short risk narrative for an engineer, three to five \
+sentences, in this order:
+ - what can go wrong: what an attacker gains if the finding is real;
+ - how an abuser attacks it: the concrete steps against THIS finding, at this url;
+ - how likely it is to lead to a breach: reason from the site's exposure (open \
+to the world raises it, allow listed lowers it), the attack surface and \
+reachability (a sensitive or unauthenticated endpoint, a path reachable with no \
+session), any compensating control visible in the evidence (authentication \
+required, input encoded, a patch already applied), and the fact that modern \
+attackers automate discovery and exploitation at scale, increasingly with AI \
+assistance, so an internet reachable and easily automated weakness (enumeration, \
+guessing, injection, or an exposed secret that bots scrape within minutes) is \
+more likely to be reached than its manual difficulty suggests.
+
+FACTS ONLY.  This is a hard rule, never break it:
+ - Use ONLY the verified real world precedent provided with the finding.  Never \
+invent or recall a CVE number, a company, a breach size, a date or a statistic.  \
+If no precedent is provided, describe the mechanism and name no incident.
+ - A precedent is a comparable case, not this site's fate.  Do not say this site \
+was breached or will be.
+ - Do not overstate.  If the exposure is allow listed, or a compensating control \
+is present, say plainly that the likelihood is lower, and why.
+ - Keep the severities and the deterministic risk as given.  Explain them; do \
+not override them.  Do not invent findings.
+
+When a SAST finding in the code supports a DAST finding (same weakness, same \
+route or parameter), say so; that raises confidence a great deal.
 
 Answer with JSON only, in this shape:
 {"summary": "three to five sentences for an engineer, plain US English",
- "verdicts": [{"id": "finding id", "verdict": "likely_real", "why": "one sentence",
+ "verdicts": [{"id": "finding id", "verdict": "likely_real", "why": "the risk narrative",
    "sast": "matching SAST finding id or empty"}],
- "priorities": ["the three things to fix first, one sentence each"]}
+ "priorities": ["the three things to fix first, worst risk first, one sentence each"]}
 
 Whenever you mention a SAST finding, give its id (GCR- and twelve characters).  \
 Whenever the summary or a priority is about a DAST finding, give its id in \
@@ -3893,8 +3919,17 @@ def triage(scan_id: int, actor: str = "") -> None:
             if links:
                 sast = _sast_lines(sast_findings(db, [l["full_name"] for l in links], 60, who))
         host = asset.host
+        exposure = (getattr(asset, "exposure", "") or "")
+        exposure_detail = (getattr(asset, "exposure_detail", "") or "")
         cap = max(1, int(settings_mod.get(db, "dast.ai_max_findings") or 40))
         prior = _prior_verdicts(db, scan.asset_id, scan_id)
+    # Deterministic risk (likelihood x breachability, weighted by exposure) and
+    # the verified real world precedent for each weakness class, computed before
+    # the model is asked, so it reasons from these facts rather than its memory.
+    from app.core import dast_precedent, dast_risk
+
+    dast_risk.annotate(all_findings, exposure)
+    dast_precedent.annotate(all_findings)
     # Reuse a verdict for any finding unchanged since the last reviewed scan.
     reused = 0
     for f in findings:
@@ -3919,11 +3954,26 @@ def triage(scan_id: int, actor: str = "") -> None:
             break
         prompt = json.dumps({
             "site": host,
+            "site_exposure": {
+                "reachable": exposure or "unknown",
+                "meaning": ("open means reachable from the open internet; "
+                            "whitelisted means only from allow listed networks; "
+                            "unknown means the exposure check did not run"),
+                "detail": _clean(exposure_detail)[:200],
+            },
             "dast_findings": [{"id": f.get("id") or f["key"], "tool": f.get("tool"),
                                "severity": f.get("severity"), "title": _clean(f.get("title")),
                                "url": _clean(f.get("url"))[:300],
                                "evidence": _clean(f.get("evidence"))[:300],
-                               "cwe": f.get("cwe")} for f in part],
+                               "cwe": f.get("cwe"),
+                               "computed_likelihood": f.get("likelihood_label"),
+                               "computed_impact": f.get("breachability_label"),
+                               "computed_risk": f.get("risk"),
+                               "risk_basis": _clean(f.get("risk_basis"))[:300],
+                               "verified_real_world_precedent": f.get("precedent") or "",
+                               "precedent_reference": f.get("precedent_ref") or "",
+                               "how_the_class_is_abused": f.get("abuse") or ""}
+                              for f in part],
             "sast_findings_in_code_behind_the_site": sast,
         }, indent=1)
         answer = ai.complete(cfg, TRIAGE_SYSTEM, prompt, purpose="dast-triage")
@@ -3954,7 +4004,7 @@ def triage(scan_id: int, actor: str = "") -> None:
             verdict = str(v.get("verdict") or "")
             f["ai_verdict"] = verdict if verdict in (
                 "likely_real", "likely_false_positive", "needs_review") else "needs_review"
-            f["ai_why"] = str(v.get("why") or "")[:400]
+            f["ai_why"] = str(v.get("why") or "")[:900]
             f["ai_sast"] = str(v.get("sast") or "")[:40]
     with session_scope() as db:
         scan = db.get(ExternalDastScan, scan_id)

app/web/routes_external.py

@@ -1283,6 +1283,11 @@ def scan_page(asset_id: int, scan_id: int, request: Request, sev: str = "",
                                                     external_decisions.for_asset(db, asset.id),
                                                     scan_id=scan.id)
         findings = view["findings"]
+        # Verified real world precedent per weakness class, from the vetted table,
+        # so a finding shows its factual comparable case even before AI triage.
+        from app.core import dast_precedent
+
+        dast_precedent.annotate(findings)
         removed = external.removed_of(scan)
         if isinstance(view["ai"].get("summary"), dict):
             view["ai"]["summary"] = external.scrub_dast_review(view["ai"]["summary"], findings,

app/web/templates/dast_report_pdf.html

@@ -246,6 +246,8 @@ <h3>{{ f.ref }}  {{ f.title }}</h3>
     {% if f.risk %}<tr><td class="k">Risk</td><td><span class="badge r-{{ f.risk }}">{{ f.risk | upper }}</span>
       &nbsp;Likelihood {{ f.likelihood_label }}, impact {{ f.breachability_label }}.
       {% if f.risk_basis %}<span class="soft">{{ f.risk_basis | sentence_spaced }}</span>{% endif %}</td></tr>{% endif %}
+    {% if f.abuse %}<tr><td class="k">How it is abused</td><td>{{ f.abuse | sentence_spaced }}</td></tr>{% endif %}
+    {% if f.precedent %}<tr><td class="k">Real-world precedent</td><td>{{ f.precedent | sentence_spaced }}  <span class="soft">({{ f.precedent_ref }})</span></td></tr>{% endif %}
     <tr><td class="k">OWASP Top 10</td><td>{{ f.owasp }}  {{ f.owasp_name }}</td></tr>
     {% if f.cwe %}<tr><td class="k">CWE</td><td>{{ f.cwe }}</td></tr>{% endif %}
     {% if f.url %}<tr><td class="k">Location</td><td class="mono">{{ f.url }}</td></tr>{% endif %}

app/web/templates/external_scan.html

@@ -200,6 +200,7 @@
             </div></details>{% endif %}
           {% if f.remediation %}<details class="small"><summary>How to fix</summary>{{ f.remediation }}</details>{% endif %}
           {% if f.ai_why %}<div class="small"><span class="badge ai">AI</span> {{ f.ai_why }}{% if f.ai_sast %} <span class="mono">{{ f.ai_sast }}</span>{% endif %}</div>{% endif %}
+          {% if f.precedent %}<details class="small"><summary>Real-world precedent</summary>{{ f.precedent }} <span class="faint">({{ f.precedent_ref }})</span></details>{% endif %}
           {% if f.reference %}<div class="small faint">{{ f.reference }}</div>{% endif %}</td>
         <td class="small mono" style="word-break:break-all">{{ f.url }}
           {% if f.rolled_up %}<details class="small"><summary>{{ f.rolled_up }} addresses</summary>{% for u in f.urls %}<div>{{ u }}</div>{% endfor %}{% if f.rolled_up > f.urls | length %}<div class="faint">and {{ f.rolled_up - f.urls | length }} more</div>{% endif %}</details>{% endif %}</td>