Skip to content

ci: fast static analysis with early-fail lint + inline SARIF (PMD/Checkstyle/SpotBugs)#1396

Draft
joaodinissf wants to merge 2 commits into
dsldevkit:masterfrom
joaodinissf:ci/static-analysis-sarif
Draft

ci: fast static analysis with early-fail lint + inline SARIF (PMD/Checkstyle/SpotBugs)#1396
joaodinissf wants to merge 2 commits into
dsldevkit:masterfrom
joaodinissf:ci/static-analysis-sarif

Conversation

@joaodinissf

@joaodinissf joaodinissf commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #1399 (restore-only publish cache); contains its commit.

Problem

Today a single PMD or Checkstyle nit only surfaces after the ~15-minute build: maven-verify runs clean verify and re-runs every static-analysis goal on top of it. So "I forgot a semicolon" feedback arrives ~14 minutes late, and the analysis work is duplicated.

What this does

Four independent parallel jobs (verify.yml), so feedback is early and findings show up inline:

Job Runs Gate
lint compile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check, -T 2C count merged SARIF (+ cpd.xml grep)
spotbugs compile + spotbugs:spotbugs (SARIF), -Xmx4g count merged SARIF
maven-verify clean verify (build + tests only) Tycho-surefire
line-endings git check shell
  • lint goes red in ~3–5 min on its own check, independent of the build — a lint issue no longer waits behind ~14 min of build+tests.
  • SpotBugs is isolated in its own lane (it's the slow analysis), so it never delays lint.
  • maven-verify drops the redundant analysis goals it used to re-run.
  • Both new lanes restore the master-produced Linux-maven-publish-* cache (restore-only, mirroring snapshot.yml's path spec and key recipe exactly — the literal path spec is hashed into the cache version, so any variation makes the producer's caches unmatchable; see ci: restore Maven cache from master-produced caches in maven-verify #1399).
  • All three analyzers emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning (security-events: write) → inline annotations on the diff + Security tab. No custom annotator script.

Why count-gate, not the :check goals

The pmd:check / spotbugs:check goals @Execute-fork a second analysis and can't emit SARIF; run without the full compile classpath they false-positive on type-resolving rules (e.g. InvalidLogMessageFormat on the SLF4J trailing-Throwable idiom). So each report goal runs once (full-reactor compile → correct results + SARIF) and the gate just counts. Full rationale, the report-vs-check tables, and the fidelity caveats are in docs/ci-static-analysis-design.md.

Optimizes / trades off

  • Optimizes: feedback latency (lint ~3–5 min vs ~14) and overall wall-clock (~10 min vs ~14.5, since the redundant analysis is gone).
  • Trades off: more total compute — compile runs once per parallel job. This is a deliberate, cheap trade here: the repo is public (CI is free) and low-activity, so we buy early + inline feedback with compute that costs nothing.

Validation

  • Clean run → all four jobs green; measured on this head rebased onto master (cache-miss runs): lint 4.0 min, spotbugs 10.8 min, maven-verify 7.4 min — wall-clock ≈ 10.8 vs 16.3 for today's monolithic job.
  • Planted one violation per tool → lint and spotbugs correctly went red (exact counts), and Code Scanning ingested PMD/Checkstyle/SpotBugs SARIF → annotations appeared on the PR diff + Security tab.

Notes

  • CPD gating is wired (pmd:cpd-check + <duplication> grep) but inert until chore: pmd.cpd.min set to 100000 effectively disables CPD #1339 lowers the token threshold (pmd.cpd.min=100000 currently disables it) — tracked separately. CPD has no SARIF renderer, so it gates but doesn't annotate inline.
  • Branch protection (admin, at merge): required checks change pmd + checkstylelint + spotbugs; keep line-endings, maven-verify.
  • See also docs/ci-measurement-protocol.md for how to gather trustworthy timing medians.

Supersedes #1333 (sequential, Python) and #1337 (parallel, Python).

🤖 Generated with Claude Code

joaodinissf and others added 2 commits July 8, 2026 18:46
The maven-verify cache never hit in practice, for three compounding reasons
(verified against the live cache inventory):

- actions/cache only restores on an exact key match, and the key embedded
  hashFiles('**/pom.xml') - so any pom-touching PR missed outright.
- GitHub isolates caches per PR scope: the open PRs each held a byte-identical
  ~700 MB cache under their own refs/pull/N/merge, invisible to one another.
  The only shareable scope is master, and verify.yml (a pull_request-only
  workflow) never saved there.
- The redundant per-PR saves pushed the repo over its 10 GB cache budget
  (10+ GB across 14 caches), so LRU eviction also killed same-PR reuse.

Fix, all in the one cache step: switch to actions/cache/restore (PRs stop
saving - ends the duplicate-cache churn and the eviction pressure) and mirror
the producer exactly. master DOES have a producer - snapshot.yml saves
~/.m2/repository as Linux-maven-publish-* on every push - so maven-verify now
adopts that family's exact path spec and key recipe. The mirroring must be
literal: GitHub hashes the path spec into the cache *version* and only serves
a cache whose key AND version both match, so ~/.m2/repository vs
/home/runner/.m2/repository alone makes the producer's caches silently
unmatchable (the repo's cache inventory showed the two path spellings under
distinct version hashes). A PR that touches no pom/target files exact-hits
master's newest cache; any other PR prefix-restores it via restore-keys.
The cache-bust lever is renaming the maven-publish family in snapshot.yml
and here together.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the pmd/checkstyle jobs with a parallel shape that gives early,
inline feedback and stops re-running analysis inside the build:

  - lint: compile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check,
    -T 2C, --fail-never; gates by counting the merged SARIF (+ cpd.xml grep).
    Fails in ~3-5 min on its own check, independent of the build.
  - spotbugs: compile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel lane
    (the slow analysis).
  - maven-verify: build + tests only; the redundant checkstyle/pmd/spotbugs
    goals are dropped (now owned by lint/spotbugs).
  - line-endings: unchanged.
  - both new lanes restore the master-produced Linux-maven-publish-* cache,
    restore-only, mirroring snapshot.yml's path spec and key recipe exactly
    (the path spec is hashed into the cache version, so the mirroring must
    be literal).

All three emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning
(security-events: write) for inline annotations on the PR diff + Security tab.
No custom Python annotator.

Count-gate rather than the *:check goals: the check goals @Execute-fork a
second analysis and cannot emit SARIF, and without the full compile classpath
they false-positive on type-resolving rules. Each report goal runs once
(full-reactor compile -> correct + SARIF) and the gate counts the result.
Rationale + tables in docs/ci-static-analysis-design.md; measurement protocol
in docs/ci-measurement-protocol.md.

CPD gating is wired but inert until dsldevkit#1339 lowers the token threshold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joaodinissf joaodinissf force-pushed the ci/static-analysis-sarif branch from fdb1272 to e8c8428 Compare July 8, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant