feat: add TruthBench semantic red-team foundation

.superpowers/sdd/task-4-report.md

@@ -0,0 +1,110 @@
+# Task 4 Report: Add finance, healthcare, retail, and CRM gold datasets
+
+## Status
+
+DONE. The four deterministic 16-row domain builders are registered and covered by focused and adjacent TruthBench tests.
+
+## TDD evidence
+
+### RED
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Observed 16 failures before implementation: each new domain was absent from the fixture registry (`FixtureError: unknown fixture domain`).
+
+### GREEN
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `28 passed`.
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py tests/truthbench/test_models_exact.py tests/truthbench/test_schema.py -q --no-cov
+```
+
+Result: all focused, model, and schema tests green.
+
+Static checks:
+
+```text
+python -m ruff check benchmarks/truthbench/fixtures tests/truthbench/test_fixtures.py
+python -m ruff format --check benchmarks/truthbench/fixtures tests/truthbench/test_fixtures.py
+mypy --ignore-missing-imports --explicit-package-bases benchmarks/truthbench/fixtures tests/truthbench/conftest.py tests/truthbench/test_fixtures.py
+git diff --check
+```
+
+All passed.
+
+## Files
+
+- `benchmarks/truthbench/fixtures/finance.py` — finance frame and oracle labels for Apple semantic traps, numeric/currency/date defects, protected ticker conflict, and synthetic PII canaries.
+- `benchmarks/truthbench/fixtures/healthcare.py` — healthcare frame and oracle labels for ICD/LOINC, temperature and dose units, FHIR/partial/impossible dates, Unicode, and PHI canaries.
+- `benchmarks/truthbench/fixtures/retail.py` — retail frame and oracle labels for leading-zero identifiers, free/return quantities, locale currency formats, mojibake/HTML, multilingual values, and review PII.
+- `benchmarks/truthbench/fixtures/crm.py` — CRM frame and oracle labels for Unicode/contact ambiguity, lifecycle contradiction, zero-width/hidden PII, Apple semantic traps, and protected IDs.
+- `benchmarks/truthbench/fixtures/__init__.py` — registry entries and stable domain order.
+- `tests/truthbench/test_fixtures.py` — domain completeness, disposition, deterministic-seed, and adversarial marker tests.
+
+## Self-review
+
+- Every builder emits exactly 16 rows with stable string indexes, fixed `2026-01-15` UTC reference metadata, explicit locale metadata, complete physical-cell labels, and at least 12 injected adversarial cells.
+- All four dispositions are represented per domain. Row cases cover exact duplicates and removed rows; schema cases cover added, removed, renamed, reordered, and type-drifted columns without mislabeling absent cells.
+- Sensitive values are whole-value synthetic `.invalid`, `TB-*`, or `555-01xx` forms, preserving the base builder's redaction and canary invariants. Zero-width examples are intentionally non-sensitive representation traps.
+- Seed values are included only in a deterministic batch marker; same-seed builds produce byte-stable serialized oracle payloads and identical fixture hashes for seeds `1729` and `2718`.
+- No FreshData runtime, LLM/provider, network, or external data dependency is used.
+
+## Concerns
+
+- The legacy `minimal` fixture remains registered for backwards compatibility; the four Task 4 domains are appended in stable order. A later registry task may choose to retire `minimal` once its callers migrate.
+- Row/schema expectations are metadata cases (the physical frame remains rectangular), consistent with the oracle contract that removed rows/columns cannot have cell labels.
+
+## Review follow-up: healthcare reference codes and content assertions
+
+### RED
+
+After review, focused content tests were strengthened to inspect each actual `GoldCell.family`, disposition, and adversarial frame value. The first run exposed four failures: the healthcare rare-code test rejected the placeholder `G rare`; finance, retail, and CRM injection-count assertions correctly counted only non-preserve dispositions rather than all injected cells.
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result before fixes: `4 failed, 24 passed`.
+
+### GREEN
+
+Healthcare preserve cases now use values present in the bundled reference sets (`Z79.4`, `F17.210`, and `9843-4`), and tests load those references plus validate ICD/LOINC syntax. Content tests assert the finance, healthcare, retail, and CRM families directly against physical cells; injected-cell counts use non-background families, including preserve injections.
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `28 passed`.
+
+Complete adjacent verification (`tests/truthbench/test_fixtures.py`, `tests/truthbench/test_models_exact.py`, and `tests/truthbench/test_schema.py`) passed. Ruff check/format, mypy, and `git diff --check` also passed.
+
+## Review follow-up: contract-gap family coverage
+
+### RED
+
+Added direct assertions for the finance USD/EUR/INR conflict and zero-width memo, healthcare protected-DOB repair conflict and MRN tail canary, and exact row/schema family sets for finance, healthcare, and CRM. The initial run exposed the missing `zero-width-memo` family label (the value existed but was grouped under the broader invisible-PII family).
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `1 failed, 30 passed` (`StopIteration` while locating the required zero-width memo family).
+
+### GREEN
+
+The zero-width memo cell now has its own `zero-width-memo` family; all assertions inspect actual frame values, dispositions, sensitivity, and exact case-family sets.
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `31 passed`.
+
+Complete fixture/models/schema verification passed (`... passed`), as did Ruff check/format, mypy, and `git diff --check`.

.superpowers/sdd/task-5-report.md

@@ -0,0 +1,85 @@
+# Task 5 Report: Complete eight-domain TruthBench corpus
+
+## Status
+
+DONE. Logistics, government, education, and insurance now have deterministic 16-row gold fixtures. The registry is the stable alphabetical eight-domain order (`crm`, `education`, `finance`, `government`, `healthcare`, `insurance`, `logistics`, `retail`) with the temporary minimal registry removed.
+
+## TDD evidence
+
+### RED
+
+After adding domain-content tests, the focused fixture suite failed with the expected missing-builder errors for all four new domains (`FixtureError: unknown fixture domain`). The failure run was:
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+### GREEN
+
+The focused suite now passes:
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `47 passed`.
+
+Adjacent fixture/model/schema verification:
+
+```text
+PYTHONPATH=src python -m pytest \
+  tests/truthbench/test_fixtures.py \
+  tests/truthbench/test_models_exact.py \
+  tests/truthbench/test_schema.py -q --no-cov
+```
+
+Result: `112 passed`.
+
+Static checks:
+
+```text
+python -m ruff check benchmarks/truthbench/fixtures tests/truthbench/test_fixtures.py
+python -m ruff format --check benchmarks/truthbench/fixtures tests/truthbench/test_fixtures.py
+mypy --ignore-missing-imports --explicit-package-bases \
+  benchmarks/truthbench/fixtures tests/truthbench/test_fixtures.py
+git diff --check
+```
+
+All passed.
+
+## Coverage
+
+- `logistics.py` covers valid UN/LOCODE-like references, kg/lb and C/F units, cross-timezone windows, 24:00 transport values, address PII, late tracking, and protected shipment IDs.
+- `government.py` covers leading-zero IDs, Indian/international grouping, fiscal/calendar ambiguity, multilingual labels, restricted national IDs, mixed legacy encoding, and retention/repair policy contradiction.
+- `education.py` covers student IDs, letter/percentage/GPA scales, school-year ambiguity, zero scores, enrollment ordering, guardian contacts, FERPA notes, and protected grade-policy conflict.
+- `insurance.py` covers policy/claim IDs, premium/reserve currency mismatch, negative reserve review, incident/report ordering, state contradiction, claimant/medical PII, and protected policy numbers.
+- All four builders emit complete physical-cell labels, all four dispositions, row duplicate/removal cases, five schema drift cases, fixed UTC/reference metadata, deterministic seed batches, and privacy-safe synthetic canaries.
+- Corpus-level tests assert all required trap categories occur across the eight domains.
+
+## Concerns
+
+None. No FreshData runtime, LLM/provider, network, or external data dependency is used.
+
+## Review follow-up: explicit contract coverage
+
+### RED
+
+The new required-family contract test intentionally used the repaired numeric output (`95`) as the adversarial frame value for education `edu-07`. The focused test failed because the actual frame value is the required raw value `"95%"` while the repair oracle separately stores `95.0` as its expected output.
+
+```text
+PYTHONPATH=src python -m pytest \
+  tests/truthbench/test_fixtures.py::test_required_domain_families_match_actual_values_and_dispositions \
+  -q --no-cov
+```
+
+Observed: one failure at `edu-07` (`'95%' != 95`).
+
+### GREEN
+
+The contract now asserts the raw value, family, disposition, and typed repair output. It also uses an explicit per-domain family mapping covering every required category (including logistics lb/F units, government IDs/grouping/language/fiscal/protected case, education scales/contacts/protected grade, and insurance IDs/grouped premium/PII/protected policy).
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_fixtures.py -q --no-cov
+```
+
+Result: `48 passed`.

.superpowers/sdd/task-6-report.md

@@ -0,0 +1,202 @@
+# Task 6 Report: Privacy-safe values and exhaustive sink scanning
+
+## Status
+
+DONE. `SinkScanner` now scans normalized canary variants across nested TruthBench
+sinks and emits only `Leak(canary_id, variant, path)` metadata. Redaction markers
+contain run-scoped HMAC-SHA256 digests and never matched text.
+
+## TDD evidence
+
+### RED
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_privacy.py -q --no-cov
+```
+
+Observed collection failure before implementation:
+
+```text
+ModuleNotFoundError: No module named 'benchmarks.truthbench.privacy'
+```
+
+### GREEN
+
+Focused privacy suite:
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_privacy.py -q --no-cov
+```
+
+Result: `18 passed`.
+
+Adjacent TruthBench suites:
+
+```text
+PYTHONPATH=src python -m pytest \
+  tests/truthbench/test_privacy.py \
+  tests/truthbench/test_fixtures.py \
+  tests/truthbench/test_models_exact.py \
+  tests/truthbench/test_schema.py -q --no-cov
+```
+
+Result: `131 passed`.
+
+Static checks:
+
+```text
+python -m ruff check benchmarks/truthbench/privacy.py \
+  benchmarks/truthbench/__init__.py tests/truthbench/test_privacy.py
+python -m ruff format --check benchmarks/truthbench/privacy.py \
+  benchmarks/truthbench/__init__.py tests/truthbench/test_privacy.py
+mypy --ignore-missing-imports --explicit-package-bases \
+  benchmarks/truthbench/privacy.py tests/truthbench/test_privacy.py
+git diff --check
+```
+
+## Review follow-up: MultiIndex and hostile-label privacy hardening
+
+### RED
+
+Added regression tests for MultiIndex columns/index level names, tuple-label
+structure during redaction, and custom non-string labels whose stringification
+contains a canary. Before the fix:
+
+```text
+pytest -q tests/truthbench/test_privacy.py --no-cov
+```
+
+Result: `24 passed, 2 failed`.
+
+The failures were the expected defects: redacting tuple labels converted them
+to lists and raised pandas `ValueError` (length mismatch), while hostile custom
+labels were not scanned.
+
+### GREEN
+
+Structure-preserving label traversal/redaction now handles tuple/MultiIndex
+levels and names, and sanitizes custom label paths before reporting or replacing
+them with digest markers:
+
+```text
+pytest -q tests/truthbench/test_privacy.py --no-cov
+```
+
+Result: `26 passed`.
+
+Adjacent TruthBench verification:
+
+```text
+pytest -q tests/truthbench --no-cov
+```
+
+Result: `139 passed`.
+
+Static checks:
+
+```text
+ruff check benchmarks/truthbench/privacy.py tests/truthbench/test_privacy.py
+ruff format --check benchmarks/truthbench/privacy.py tests/truthbench/test_privacy.py
+mypy benchmarks/truthbench/privacy.py
+git diff --check
+```
+
+All passed.
+
+All passed.
+
+## Files
+
+- `benchmarks/truthbench/privacy.py` — `Leak`, `PrivacySafeValue`, named
+  normalizers, run-scoped HMAC scanner, recursive redaction, typed redaction
+  handling, pandas/dataclass/bytes support, self-test, and named sink entry points.
+- `benchmarks/truthbench/__init__.py` — exports privacy scanner primitives.
+- `tests/truthbench/test_privacy.py` — mutation matrix for every required
+  normalized form, nested sink coverage, redaction/self-test behavior, and typed
+  redaction digest safety.
+
+## Self-review
+
+- Leak objects contain only identifiers, transform labels, and JSONPath-like
+  locations; `repr(leaks)` cannot repeat canary text.
+- Literal, case-folded, whitespace-stripped, punctuation-stripped, digit-only,
+  URL-decoded, HTML-unescaped, UTF-8/hex/escape bytes, NFKC/NFC/NFD,
+  zero-width-removed, and JSON-escaped forms are covered.
+- Mapping, sequence, dataclass, pandas DataFrame/Series/Index, exception,
+  report, plan, generated-code, stream, markup, JSON, and failure-artifact sinks
+  are traversed. Exact redacted `TypedValue` payloads are treated as safe and
+  their HMAC digests are not re-scanned as plaintext.
+- Redaction is recursive, emits `[REDACTED:<digest>]`, and `self_test` raises on
+  an unredacted result while accepting the scanner's own redacted output.
+
+## Concerns
+
+- The scanner intentionally treats digit-only normalized forms conservatively;
+  very short numeric canaries can match unrelated text. Fixtures use synthetic
+  identifiers and email/phone canaries, so this does not affect the bundled
+  corpus.
+- Redacting a pandas object may change a sensitive column to object/string dtype,
+  which is preferable to retaining a raw value in an audit sink.
+
+## Follow-up RED/GREEN evidence
+
+A follow-up regression test used the exact sensitive `TypedValue` redaction shape
+with a one-digit canary. Before the redacted-payload guard, the digest's hex text
+was incorrectly reported as a digit-only leak. After adding the guard:
+
+```text
+PYTHONPATH=src python -m pytest \
+  tests/truthbench/test_privacy.py::test_scanner_accepts_exact_typed_redaction_without_scanning_digest \
+  -q --no-cov
+```
+
+Result: `1 passed`.
+
+## Review follow-up: marker, key, and label hardening
+
+### RED
+
+The review regression matrix was run before the hardening changes:
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_privacy.py -q --no-cov
+```
+
+Result: `4 failed, 17 passed` for forged redaction markers, sensitive mapping
+keys, pandas labels, and the digest compatibility alias.
+
+### GREEN
+
+After validating markers against the scanner's own 64-hex HMAC digest set,
+redacting mapping keys and pandas column/index/name labels, scanning arbitrary
+key stringifications, and adding `digest` as an alias:
+
+```text
+PYTHONPATH=src python -m pytest tests/truthbench/test_privacy.py -q --no-cov
+```
+
+Result: `23 passed`.
+
+Adjacent TruthBench verification:
+
+```text
+PYTHONPATH=src python -m pytest \
+  tests/truthbench/test_privacy.py \
+  tests/truthbench/test_fixtures.py \
+  tests/truthbench/test_models_exact.py \
+  tests/truthbench/test_schema.py -q --no-cov
+```
+
+Result: `136 passed`.
+
+Static checks were rerun after the follow-up and remained clean:
+
+```text
+python -m ruff check benchmarks/truthbench/privacy.py \
+  tests/truthbench/test_privacy.py
+python -m ruff format --check benchmarks/truthbench/privacy.py \
+  tests/truthbench/test_privacy.py
+mypy --ignore-missing-imports --explicit-package-bases \
+  benchmarks/truthbench/privacy.py tests/truthbench/test_privacy.py
+git diff --check
+```