Fix four issues

docs/AST_PARSER_FALLBACK.md

@@ -0,0 +1,115 @@
+# AST Parser Fallback Strategy
+
+A single parser failing should not cost a whole file. New syntax, a half-edited file, a parser bug
+or a timeout can all make the preferred parser reject input that an analyzer could still use in
+part. The fallback chain tries parser strategies in order for each file, returns the best tree it
+can get, and records exactly what happened, so degraded results are visible and never silent.
+
+| Layer | Location |
+| --- | --- |
+| Fallback chain and lexical outline (pure, engine) | `libs/engine/core/ast-parser-fallback.ts` |
+| Fixtures | `tests/fixtures/ast-parser-fallback/` |
+| Tests | `libs/engine/core/ast-parser-fallback.spec.ts` |
+
+## Public interface
+
+```ts
+import {
+  parseWithFallback,
+  createLexicalOutlineStrategy,
+  TransientParserError,
+} from '@engine/core/ast-parser-fallback';
+
+const result = await parseWithFallback(
+  [{ path: 'contracts/Vault.sol', content: '...' }],
+  [strictSolidityParser, recoveringSolidityParser, createLexicalOutlineStrategy()],
+  { timeoutMs: 5000, requireComplete: false },
+);
+
+for (const file of result.files) {
+  if (file.status === 'degraded') {
+    // file.ast is usable but file.complete may be false; file.attempts says why.
+  }
+}
+```
+
+A strategy is `{ name, languages, maxBytes?, parse(source, { path, language }) }`. `parse` may be
+sync or async and returns `{ ast, complete?, diagnostics? }`. `complete: false` marks a partial
+tree (a recovering parser, an outline). Register strategies most faithful first.
+
+## Selection rules
+
+1. The first strategy returning a complete tree wins. The file is `parsed` if that was the first
+   strategy tried, `degraded` if an earlier one failed (`fallbackUsed: true`).
+2. A partial tree is remembered and the chain continues in case a later strategy completes.
+3. If none completes, the earliest partial tree is returned as `degraded` (`complete: false`).
+   With `requireComplete: true` the file is `failed` instead, for gates that need a full tree.
+4. A file whose language no strategy supports is `unsupported`; a file larger than `maxFileBytes`
+   is `skipped`; if every strategy fails the file is `failed`. None of these produce an AST, so an
+   analyzer can never mistake them for an empty file.
+
+`strategyOrder` overrides the order per language by strategy name. `disabledStrategies` removes
+strategies without changing the registered list.
+
+## Built-in lexical outline
+
+`createLexicalOutlineStrategy()` never fails on text input. For Solidity, Rust and Vyper it blanks
+comments and strings, matches braces (or indentation for Vyper) and returns an `OutlineAst` of
+declarations with line ranges: top-level contracts, functions, structs, enums, events, errors,
+traits, modules and constants, plus functions and events inside contracts, interfaces, libraries,
+`impl`, `trait` and `mod` blocks. It always reports `complete: false` with a `lexical_outline`
+warning, plus `unterminated_block` / `unmatched_close_brace` errors for unbalanced files.
+Its default `maxBytes` (20 MiB) is higher than typical parsers', so large files that strict
+parsers skip still get an outline.
+
+## Configuration
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `timeoutMs` | `5000` | Budget per attempt. |
+| `maxAttemptsPerStrategy` | `2` | Attempts for *transient* errors only. |
+| `maxFileBytes` | 5 MiB | Larger files are `skipped`. |
+| `maxDiagnosticsPerFile` | `50` | Diagnostics beyond this are dropped. |
+| `requireComplete` | `false` | Fail instead of returning a partial tree. |
+| `strategyOrder` | none | `{ [language]: [strategy names] }`. |
+| `disabledStrategies` | `[]` | Strategy names to skip. |
+| `languageOf` | by extension | Custom path to language mapping. A source can also set `language`. |
+| `rootDir` | none | Absolute root used to relativize absolute input paths. |
+
+## Failure semantics
+
+| Situation | Behavior |
+| --- | --- |
+| Strategy throws | Attempt `error`; next strategy. Not retried: parsers are deterministic. |
+| `TransientParserError` (or `error.transient === true`) | Attempt `transient-error`; retried up to `maxAttemptsPerStrategy`. |
+| Strategy exceeds `timeoutMs` | Attempt `timeout`; next strategy. A synchronous parser cannot be interrupted, so its result is discarded. |
+| Strategy returns no `ast` | Attempt `invalid-output`; next strategy. |
+| File larger than a strategy's `maxBytes` | Attempt `skipped-size`; next strategy. |
+| Invalid path, duplicate path or duplicate strategy name | File or strategy dropped, reported in `issues`. |
+| Invalid option or unknown name in `strategyOrder` | Default used or name ignored, `INVALID_CONFIG` / `UNKNOWN_STRATEGY`. |
+
+`parseWithFallback` never throws for bad input. Error text is truncated to one line (200 chars) and
+diagnostics are reduced to code, line, column and message, so file content is not copied into
+results. Diagnostics map directly onto `ParserRecoveryDiagnostics` (`apps/api/src/scanner`).
+
+## Determinism, retries and tenancy
+
+- Files are processed in path order, one at a time. The result depends only on sources, strategies
+  and options, never on input order. Run-dependent data lives in `telemetry`. A timeout is the one
+  machine-dependent outcome; it is visible per file in `attempts` and counted in `summary.timedOut`.
+- Retrying a whole job is safe; there is no module state or I/O. Cache by commit, options and the
+  versions of the strategies in the chain.
+- Strategies are supplied per call, so each tenant or job can have its own chain, and the chain
+  only sees the sources passed in.
+
+## Using the result
+
+`status`, `complete` and `strategy` tell an analyzer how much to trust the tree. Rules that need
+full syntax should skip `complete: false` files and say so; rules that work from declarations can
+use an outline. A common policy is to treat findings from degraded files with lower confidence,
+for example with a `ruleOverrides` entry in rule-level confidence calibration.
+
+## Backward compatibility
+
+Additive. Existing analyzers and parsers are untouched; a parser joins the chain by being wrapped
+as a strategy. Without a chain, behavior is unchanged.

docs/CROSS_FILE_REFERENCE_RESOLUTION.md

@@ -0,0 +1,114 @@
+# Cross-File Reference Resolution
+
+Rules that look at one file at a time cannot tell where an imported symbol comes from. Cross-file
+resolution builds the import graph for a repository snapshot, indexes top-level definitions, and
+resolves each use of an imported symbol to the file and line that defines it.
+
+| Layer | Location |
+| --- | --- |
+| Resolver (pure, engine) | `libs/engine/core/cross-file-resolver.ts` |
+| Fixtures | `tests/fixtures/cross-file-resolution/` |
+| Tests | `libs/engine/core/cross-file-resolver.spec.ts` |
+
+## Public interface
+
+```ts
+import {
+  resolveCrossFileReferences,
+  getAffectedFiles,
+  getReferencesTo,
+  parseRemappings,
+} from '@engine/core/cross-file-resolver';
+
+const resolution = resolveCrossFileReferences(
+  [{ path: 'contracts/Vault.sol', content: '...' }, /* ... */],
+  { remappings: parseRemappings(remappingsTxt) },
+);
+```
+
+- `resolveCrossFileReferences(sources, options)` returns a `CrossFileResolution`
+  (`schemaVersion`, `files`, `skippedFiles`, `symbols`, `imports`, `references`, `dependencies`,
+  `cycles`, `issues`, `counts`).
+- `getAffectedFiles(resolution, changedFiles)` returns the changed files plus all transitive
+  importers. Use it to widen `AnalyzerConfig.changedFiles` when `changedFilesOnly` is set, so a
+  change to a base contract re-analyzes its dependents.
+- `getReferencesTo(resolution, file, name)` lists every use that resolves to a definition.
+- `parseRemappings(text)` parses a `remappings.txt` body.
+
+## Configuration
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `remappings` | `{}` | Solidity `prefix -> target` remappings. The longest matching prefix wins. |
+| `rootDir` | none | Absolute root used to relativize absolute input paths. |
+| `maxFiles` | `5000` | Files beyond this count (after sorting by path) are skipped with `TOO_MANY_FILES`. |
+| `maxFileBytes` | `1048576` | Larger files are skipped with `FILE_TOO_LARGE`. |
+
+There are no other switches. Languages are chosen by extension.
+
+## Supported languages
+
+| Language | Extensions | Imports understood | Symbols indexed (top level) |
+| --- | --- | --- | --- |
+| Solidity | `.sol` | `import "x"`, `import "x" as N`, `import * as N from "x"`, `import {A, B as C} from "x"`, remappings | contract, interface, library, struct, enum, event, error, function, user-defined type, constant |
+| Rust | `.rs` | `mod x;`, `use crate::/self::/super::` paths, `{}` groups, `as`, `*`, `pub use` / `pub mod` re-exports | fn, struct, enum, trait, type, const, static, union |
+| Vyper | `.vy`, `.vyi` | `import a.b as c`, `from a import B`, `from . import m`, relative dots | def, struct, event, interface, flag, enum, constants |
+
+Solidity plain imports (`import "x";`) and symbol imports are transitively visible to importers, as
+in the Solidity compiler. Rust and Vyper imports are not re-exported unless marked `pub use`.
+
+## Output semantics
+
+- **Imports** have a `status`:
+  - `resolved`: the target is in the snapshot (`resolvedPath` is set).
+  - `external`: the specifier points outside the snapshot (`@openzeppelin/...` with no vendored
+    copy, `std::...`, `ethereum.ercs`). Not an error.
+  - `unresolved`: a relative import, `mod`, or in-repo `use` whose target is missing. Raises
+    `UNRESOLVED_IMPORT`.
+- **References** have a `status`:
+  - `resolved` with a single `target` definition.
+  - `ambiguous` with all `candidates`, when two plain imports provide the same name
+    (`AMBIGUOUS_SYMBOL`).
+  - `external` when the symbol comes from an external import.
+  - `unresolved` when the import resolved but defines no such symbol (`UNRESOLVED_SYMBOL`).
+- Comments and string contents are never counted as references. A definition in the same file
+  shadows an import of the same name. `a.b` member accesses are not treated as imports of `b`.
+- Namespace uses (`Errors.ZeroAmount`, `math::sub`) are reported with the member as `name` and the
+  namespace as `via`.
+- `cycles` lists strongly connected import groups; each also raises one `IMPORT_CYCLE` issue.
+  Resolution terminates on cycles.
+
+## Failure semantics
+
+| Situation | Behavior |
+| --- | --- |
+| Unsupported extension | File listed in `skippedFiles`, issue `UNSUPPORTED_LANGUAGE`. |
+| Path is empty, absolute outside `rootDir`, or escapes the root (`..`) | File dropped, `INVALID_PATH`. Relative imports that climb above the root are `unresolved`. |
+| Duplicate path | One copy kept (smallest content, so input order does not matter), `DUPLICATE_PATH`. |
+| File too large / too many files | Skipped, `FILE_TOO_LARGE` / `TOO_MANY_FILES`. |
+| Exception while processing one file | `PARSE_ERROR` for that file; every other file is still resolved. |
+| Empty input | Empty resolution with no issues. |
+
+The resolver never throws for bad repository input and never returns a partial result silently:
+every skipped or degraded item has an entry in `issues`.
+
+## Determinism, retries and tenancy
+
+- The function is pure: no I/O, clock, randomness or module state. For the same sources and
+  options it returns byte-identical output, whatever order the sources arrive in. Every list is
+  sorted (by path, line, column, name).
+- A worker can retry a failed job freely; there is nothing to clean up. Persist or cache results
+  keyed by commit SHA, `options` and rule-pack version.
+- It only sees the sources passed in, so tenants are isolated as long as one call is made per
+  tenant repository. Paths in the output are repository-relative.
+
+## Backward compatibility
+
+The module is additive. No existing interface changed, analyzers that ignore it behave exactly as
+before, and `schemaVersion` is bumped when the output shape changes.
+
+## Limits
+
+Resolution is lexical, not a full parse: names inside function bodies are matched by identifier, so
+a local variable that shares a name with an imported symbol is counted as a reference. Contract
+members, inline Rust modules and Solidity `using ... global` are not indexed.

docs/RULE_CONFIDENCE_CALIBRATION.md

@@ -0,0 +1,97 @@
+# Rule-Level Confidence Calibration
+
+Some rules are right almost every time, others mostly produce noise. Calibration turns reviewed
+outcomes (true / false positive verdicts) into a per-rule track record and uses it to attach a
+confidence value and band to each finding, so reports and gates can rank or filter by how much a
+finding is likely to be real.
+
+| Layer | Location |
+| --- | --- |
+| Calibrator (pure, engine) | `libs/engine/core/confidence-calibration.ts` |
+| Fixtures | `tests/fixtures/confidence-calibration/` |
+| Tests | `libs/engine/core/confidence-calibration.spec.ts` |
+
+## Public interface
+
+```ts
+import {
+  buildCalibrationProfiles,
+  calibrateFindings,
+  confidenceBand,
+} from '@engine/core/confidence-calibration';
+
+// 1. Aggregate one tenant's reviewed outcomes for the active rule pack.
+const { profiles } = buildCalibrationProfiles(outcomes, { rulePackVersion: '2.1.0' });
+
+// 2. Calibrate a scan's findings (normalized or raw; anything with a `ruleId`).
+const result = calibrateFindings(findings, { profiles, rulePackVersion: '2.1.0', minConfidence: 0.3 });
+result.findings;   // kept, in input order, each with metadata.confidence / confidenceBand / ...
+result.suppressed; // below the threshold, still carrying their confidence
+result.issues;     // every ignored profile, bad value or fallback
+```
+
+Each returned finding gets these `metadata` fields: `confidence` (0 to 1, 4 decimals),
+`confidenceBand` (`high` / `medium` / `low`), `confidenceSource`, `calibrationSamples`, and
+`rawConfidence` when the finding supplied one in `metadata.confidence`.
+
+## Model
+
+For a rule with `tp` true and `fp` false positives, prior `p` and prior strength `s`:
+
+```
+q          = (tp + p*s) / (tp + fp + s)                      smoothed true-positive rate
+confidence = sigmoid(logit(r) + logit(q) - logit(p))         r = raw confidence, or p if absent
+```
+
+With no raw confidence the result is `q`. With one, the raw value is shifted by how far the rule's
+record is from its prior, so an accurate rule raises it and a noisy rule lowers it, and the result
+always stays in (0, 1). Rules with fewer than `minSamples` reviewed outcomes are not calibrated: the
+finding keeps its raw confidence (`confidenceSource: "raw"`) or the prior (`"prior"`).
+
+Worked example (prior 0.5, strength 10): 45 true and 5 false positives give `q = 50/60 = 0.8333`.
+A finding with raw confidence 0.9 has odds 9, the rule's odds are 5, and the result is `45/46 = 0.9783`.
+
+## Configuration
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `profiles` | `[]` | Per-rule `{ ruleId, truePositives, falsePositives, priorConfidence?, priorStrength?, rulePackVersion? }`. |
+| `rulePackVersion` | none | When set, profiles recorded against another version are ignored (`STALE_PROFILE`). |
+| `defaultPrior` | `0.5` | Prior for rules without their own. |
+| `priorStrength` | `10` | Prior weight, in pseudo-observations. |
+| `minSamples` | `5` | Reviewed outcomes needed before calibration applies. |
+| `minConfidence` | `0` | Findings below this are moved to `suppressed`. `0` suppresses nothing. |
+| `ruleOverrides` | `{}` | `{ [ruleId]: { confidence?, minConfidence? } }`. `confidence` pins the value (`source: "override"`); `minConfidence` replaces the global threshold for that rule. |
+| `bandThresholds` | `{ high: 0.8, medium: 0.5 }` | Inclusive lower bounds of the bands. Must satisfy `0 <= medium < high <= 1`. |
+
+## Failure semantics
+
+The calibrator never throws for bad data; it ignores the bad part, falls back to defaults and
+records an issue.
+
+| Situation | Behavior |
+| --- | --- |
+| Profile with bad counts, prior or strength | Ignored, `INVALID_PROFILE`. The rule is treated as having no history. |
+| Profile for another rule-pack version | Ignored, `STALE_PROFILE`. |
+| Several profiles for one rule | The one with the most samples is kept, `DUPLICATE_PROFILE`. |
+| Raw `metadata.confidence` outside [0, 1] or not a number | Ignored, `INVALID_CONFIDENCE`. |
+| Finding without a `ruleId` | Gets the default prior, `MISSING_RULE_ID`. |
+| Invalid config value | Default used, `INVALID_CONFIG`. |
+| Malformed review outcome | Skipped, `INVALID_OUTCOME`. |
+| Unknown rule or no outcomes | Not an error: raw confidence or prior, `source` says which. |
+
+## Determinism, retries and tenancy
+
+- Pure function: no I/O, clock or state. The same findings, profiles and config always give
+  identical output, regardless of profile order. Retrying a failed job is safe.
+- Idempotent: calibrating findings that were already calibrated gives the same result, because the
+  original raw confidence is kept in `rawConfidence`.
+- Tenant isolation is by construction: only the outcomes and profiles passed in are used. Build
+  profiles from one tenant's reviews and cache them per tenant, rule-pack version and review window.
+- Persist profiles (or the outcomes) with the rule-pack version they were computed for. Reproducing
+  a result needs the same commit, config and rule-pack version.
+
+## Backward compatibility
+
+Additive. Without config, findings only gain `confidence*` metadata; nothing is suppressed and no
+existing field changes. Consumers that do not read the new metadata see no difference.