Skip to content

feat: add golden output snapshots for supported languages (#1253) - #1286

Merged
mijinummi merged 1 commit into
MDTechLabs:mainfrom
p3ris0n:feat/add-golden-output-snapshots-for-supported-languages-1253
Oct 20, 2026
Merged

mijinummi merged 1 commit into
MDTechLabs:mainfrom
p3ris0n:feat/add-golden-output-snapshots-for-supported-languages-1253

Conversation

@p3ris0n

@p3ris0n p3ris0n commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #1253
Closes #1249
Closes #1248
Closes #1247

This PR extends GasGuard's repository analysis platform across both the Quality and Evaluation and API and Integrations workstreams. It introduces golden output snapshot verification across supported languages, a labeled benchmark corpus for analyzer rules, API deprecation and sunset policy enforcement with lifecycle headers, and end-to-end request correlation identifiers across API, worker, and RPC boundaries.


Key Implementations

1. Golden Output Snapshots for Supported Languages (Closes #1253)

  • Deterministic Evaluator: Canonical finding sorting (file -> startLine -> startColumn -> ruleId -> message) and deterministic 16-char hex fingerprinting across multi-language scans (Solidity, Soroban Rust, Vyper, Cairo, TypeScript).
  • Multi-Language Diff Engine: Computes structured diffs (added, removed, modified) across finding lists, files analyzed, and severity summaries.
  • Tenant Isolation & Scan Pinning: Storage partitioned strictly by tenantId; write-once scan pins preserve the comparison snapshot across retries.
  • Fixtures & Documentation: Added realistic multi-language repository fixture (realistic-repo-fixture.json), regression test fixture (regression-case-fixture.json), and documentation in docs/GOLDEN_OUTPUT_SNAPSHOTS.md.

2. Labeled Benchmark Corpus for Analyzer Rules (Closes #1249)

  • Corpus Schema & Catalog: Established canonical benchmark dataset schema with ground-truth labeled findings (true positives, true negatives, gas savings annotations, and CWE/SWC vulnerability mappings).
  • Benchmark Evaluator: Calculates precision, recall, F1 score, and false positive rates per rule and language under fixed rule-pack versions.
  • Deterministic Baselines: Regression evaluation compares rule revisions against corpus baselines to detect drop in detection quality or surge in false positives.
  • Fixtures & Documentation: Included representative rule corpus suites and complete runbook in docs/ANALYZER_BENCHMARK_CORPUS.md.

3. API Deprecation and Sunset Policy (Closes #1248)

  • Lifecycle Headers Middleware: Implements standard HTTP deprecation headers (Deprecation: @<timestamp>, Sunset: <date>, and Link: <replacement>; rel="successor-version").
  • Policy Registry & Governance: Structured, versioned deprecation schedules preventing sudden sunset transitions without mandatory notice periods.
  • Telemetry & Alerts: Emits bounded Prometheus metrics (gasguard_api_deprecated_route_requests_total) and structured warnings for client migration tracking.
  • Contract & Documentation: Published formal API deprecation and sunset policy in docs/API_DEPRECATION_POLICY.md.

4. API Request Correlation Identifiers (Closes #1247)

  • Correlation Middleware: Ingests incoming X-Correlation-ID / X-Request-ID headers or generates standardized UUIDv4 correlation IDs on ingress.
  • Propagation Boundary: Injects correlation identifiers into AsyncLocalStorage context, outgoing HTTP/RPC client calls, and background queue job metadata (scanId, correlationId).
  • Sanitized Structured Logging: Automatically attaches correlationId to all log records while strictly redacting auth headers, tokens, private keys, and passwords.
  • Egress Header: Mirrors X-Correlation-ID in all API responses to simplify client-side tracing and support debugging.

Architectural & Security Compliance

  • Tenant Isolation: All caches, storage paths (<baseDir>/<sanitized_tenantId>/...), and evaluation contexts are isolated by tenantId.
  • Zero Secrets in Logs: Context sanitization scrubs tokens (ghp_*), JWTs, private keys, and repository source payloads.
  • Determinism: Identical commits, configurations, and rule-pack versions yield byte-identical evaluation fingerprints.

Verification Evidence & Test Coverage

  • Unit Tests:
    • golden-snapshot-evaluator.spec.ts (canonicalization, hashing, diffing)
    • rule-benchmark-evaluator.spec.ts (precision/recall calculations, baseline checks)
    • api-deprecation.middleware.spec.ts (header injection, sunset expiration dates)
    • correlation-id.middleware.spec.ts (header validation, generation, propagation)
  • Integration Tests:
    • golden-snapshot.integration.spec.ts (file-store persistence, worker task execution)
    • correlation-id.integration.spec.ts (end-to-end trace propagation across API -> Queue -> Worker)
  • Fixtures: Verified against multi-language DEX repositories and regression suites.

@drips-wave

drips-wave Bot commented Oct 19, 2026

Copy link
Copy Markdown

@p3ris0n Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@mijinummi
mijinummi merged commit a02bdc4 into MDTechLabs:main Oct 20, 2026
10 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants