Skip to content

feat: implement mapping integrity validator for CWE and MITRE ATLAS - #147

Open
rashim27us wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rashim27us:feature/mapping-integrity-validator
Open

rashim27us wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rashim27us:feature/mapping-integrity-validator

Conversation

@rashim27us

Copy link
Copy Markdown

Summary

This PR adds the first phase of automated cross-framework mapping integrity validation for the 2026 release, starting with CWE 4.20 and MITRE ATLAS 2026.06 as requested in #132.

  • Adds a local and CI validator that checks mapping catalogs against version-pinned primary datasets, including identifier existence, duplicate IDs, title/metadata consistency, source version checks, pinned ATLAS source enforcement, and SHA-256 verification via new source_sha fields.
  • Updates cwe-4.20.json and mitre-atlas-2026.06.json to pin machine-readable source URLs and record the verified source digests, replacing the CWE homepage URL with the version-specific XML archive.
  • Adds offline negative tests, a GitHub Actions workflow, and mapping documentation so future framework validators can follow the same pattern.

Test plan

  • Run python .github/scripts/test_validate_mappings.py
  • Run python .github/scripts/validate_mappings.py --download
  • Confirm both 2026/final/mappings/cwe-4.20.json and 2026/final/mappings/mitre-atlas-2026.06.json pass validation
  • Verify negative cases fail for invalid IDs, duplicate IDs, title mismatches, deprecated ATLAS source paths, version mismatches, and SHA mismatches

@avp9-nexus

Copy link
Copy Markdown

I ran the seven files of this PR at ce3efca in a sandbox, since the two notes from #132 are what it implements.

What holds, measured. The 8 offline tests pass with no network. With the network on, both catalogs pass, and the two fetched sources hash to the recorded source_sha values (ATLAS at 625,790 bytes, the size the mapping notes already give for the July read). As a control on the real file, I set the title of AML.TA0000 in mitre-atlas-2026.06.json to the stale "ML Model Access": the validator fails with a title mismatch, and passes again once the file is restored. So the stale default path and the unpinned bytes are both closed.

One thing does not match the premise of #132, "avoiding dependence on live external APIs during CI". The workflow runs validate_mappings.py --download, .cache/ is ignored and no step restores it, so every run fetches both sources. With the network off and an empty cache, that command exits 1 and prints FAIL for both catalogs with "could not download". An unreachable host and an invalid mapping get the same verdict and the same exit code, so an outage at cwe.mitre.org would turn every PR that touches the mappings red.

Two small ways out. A separate outcome for "source unavailable", with its own message and exit code, so CI can tell "not checked" from "wrong". Or a cache step for .cache/mapping-sources, so runs reuse the pinned bytes: the digest check already makes a cached copy safe to trust. The first one fixes the meaning, the second only makes it rarer.

Avp9.

@drk-ram drk-ram left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all seven changed files at ce3efcae77a7006937f48b46ad7f14ac5e052ec8. Approving this explicitly scoped first phase.

The important integrity boundary is implemented correctly: validate_catalog() checks the exact source-byte SHA-256 before interpreting source metadata or accepting identifiers. The version-specific ATLAS v6 path avoids the deprecated default dataset, and CWE names include their abstraction rather than accepting an identifier alone. The read-only workflow token and offline negative fixtures are appropriate.

Validation:

  • python .github\scripts\test_validate_mappings.py — passed, 8 tests.
  • python .github\scripts\validate_mappings.py --download — passed for both actual catalogs, including their published source hashes, versions, identifiers, and titles.

Non-blocking hardening: .github/scripts/validate_mappings.py:254-281 continues into the catalog-specific validator after structural validation has already found malformed elements. I reproduced an element with id: []: validate_common() records the expected diagnostic, then validate_cwe() raises TypeError: unhashable type: 'list'. This still fails closed, so it is not an acceptance bypass, but it loses the useful report and prevents checking the remaining catalog. Please consider short-circuiting structural errors and adding cases for non-array elements and non-string IDs.

Scope matters: these checks establish catalog integrity, not the semantic correctness or completeness of every cross-framework relationship. Keep that distinction explicit as #132 expands. For future signed mapping receipts, the canonical-preimage rules in draft-durga-agenttrust-trace-verification-00 §3 (work in progress) and TR-2026-03, DOI: 10.5281/zenodo.23775009 are relevant. They are separate from this PR's correct use of hashes over the original MITRE source bytes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants