ci: in-process SpotBugs + changed-modules-only reactor in the spotbugs lane#1456
Draft
joaodinissf wants to merge 5 commits into
Draft
ci: in-process SpotBugs + changed-modules-only reactor in the spotbugs lane#1456joaodinissf wants to merge 5 commits into
joaodinissf wants to merge 5 commits into
Conversation
The maven-verify cache never hit in practice, for three compounding reasons
(verified against the live cache inventory):
- actions/cache only restores on an exact key match, and the key embedded
hashFiles('**/pom.xml') - so any pom-touching PR missed outright.
- GitHub isolates caches per PR scope: the open PRs each held a byte-identical
~700 MB cache under their own refs/pull/N/merge, invisible to one another.
The only shareable scope is master, and verify.yml (a pull_request-only
workflow) never saved there.
- The redundant per-PR saves pushed the repo over its 10 GB cache budget
(10+ GB across 14 caches), so LRU eviction also killed same-PR reuse.
Fix, all in the one cache step: switch to actions/cache/restore (PRs stop
saving - ends the duplicate-cache churn and the eviction pressure) and mirror
the producer exactly. master DOES have a producer - snapshot.yml saves
~/.m2/repository as Linux-maven-publish-* on every push - so maven-verify now
adopts that family's exact path spec and key recipe. The mirroring must be
literal: GitHub hashes the path spec into the cache *version* and only serves
a cache whose key AND version both match, so ~/.m2/repository vs
/home/runner/.m2/repository alone makes the producer's caches silently
unmatchable (the repo's cache inventory showed the two path spellings under
distinct version hashes). A PR that touches no pom/target files exact-hits
master's newest cache; any other PR prefix-restores it via restore-keys.
The cache-bust lever is renaming the maven-publish family in snapshot.yml
and here together.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the pmd/checkstyle jobs with a parallel shape that gives early,
inline feedback and stops re-running analysis inside the build:
- lint: compile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check,
-T 2C, --fail-never; gates by counting the merged SARIF (+ cpd.xml grep).
Fails in ~3-5 min on its own check, independent of the build.
- spotbugs: compile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel lane
(the slow analysis).
- maven-verify: build + tests only; the redundant checkstyle/pmd/spotbugs
goals are dropped (now owned by lint/spotbugs).
- line-endings: unchanged.
- both new lanes restore the master-produced Linux-maven-publish-* cache,
restore-only, mirroring snapshot.yml's path spec and key recipe exactly
(the path spec is hashed into the cache version, so the mirroring must
be literal).
All three emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning
(security-events: write) for inline annotations on the PR diff + Security tab.
No custom Python annotator.
Count-gate rather than the *:check goals: the check goals @Execute-fork a
second analysis and cannot emit SARIF, and without the full compile classpath
they false-positive on type-resolving rules. Each report goal runs once
(full-reactor compile -> correct + SARIF) and the gate counts the result.
Rationale + tables in docs/ci-static-analysis-design.md; measurement protocol
in docs/ci-measurement-protocol.md.
CPD gating is wired but inert until dsldevkit#1339 lowers the token threshold.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SpotBugs' per-module analysis is the spotbugs job's long pole. A PR only needs its changed modules scanned, so a pre-step injects <spotbugs.skip>true> into every unchanged reactor module's pom — the plugin then skips the goal, and the per-module JVM fork, for them. The full-reactor compile is kept (a changed module keeps its complete aux-classpath); a build/config change falls back to a full scan. pull_request only — master/snapshot run a full scan. -Dspotbugs.onlyAnalyze was the cleaner-looking alternative but screens too late (after the per-module fork), ~17% vs ~88% measured; the script header documents the migration if an upstream SpotBugs early-exit ever lands. - .github/scripts/compute-spotbugs-skip.sh: diff -> changed modules -> inject skip into the unchanged ones (idempotent; build/config change -> full scan). - verify.yml spotbugs job: fetch-depth 0 + a scope step before compile; -Djgit.dirtyWorkingTree=ignore because the scope step dirties poms on purpose and this job releases nothing (releases/maven-verify keep =error); SARIF upload guarded so an empty scan set (no module scanned) doesn't fail the upload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spotbugs-maven-plugin forks a fresh JVM per module by default (maxHeap 2048), so a full scan pays 59 JVM startups. -Dspotbugs.fork=false runs the analysis inside the Maven JVM; the lane's MAVEN_OPTS -Xmx4g governs the shared heap (the plugin's maxHeap setting applies to forks only). Measured locally (full scan, clean tree, JDK 21): 75s vs 95s wall at -T 3C, and stable at CI thread width (-T 8, 4 GB heap) with identical findings (59 module SARIFs, same result count). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compute-spotbugs-skip.sh now also exports SPOTBUGS_SCOPE_ARGS
("-pl <changed modules> -am") and the spotbugs lane passes it to mvn,
so a scoped run builds only the PR's changed modules plus their
upstream dependencies instead of the full 64-module reactor. The
-am-pulled unchanged dependencies keep the injected spotbugs.skip:
they compile (complete aux-classpath) but are not analysed.
Fail-safes are unchanged: a build/config change means a full reactor
and full scan, and a PR touching no reactor module builds the full
reactor with every analysis skipped.
Measured locally (single-module change, clean tree, JDK 21): 23.5s
wall vs 75s for the full-reactor equivalent, with dependencies
resolved from the scoped reactor (no repository fallback).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on #1400. Earlier commits belong to #1399/#1396/#1400; review only the last two commits here.
Two independent reductions to the
spotbugslane, the verify pipeline's critical path (~8.5 min full scan):-Dspotbugs.fork=false— the plugin forks a fresh 2 GB JVM per module by default (59 forks per full scan). Analysis now runs inside the Maven JVM; the lane'sMAVEN_OPTS: -Xmx4ggoverns the shared heap. Findings are identical (59 module SARIFs, same result count).compute-spotbugs-skip.shadditionally exportsSPOTBUGS_SCOPE_ARGS(-pl <changed modules> -am), so a scoped run builds only the PR's changed modules plus their upstream dependencies instead of the full 64-module reactor. The-am-pulled unchanged dependencies keep the injectedspotbugs.skip: they compile (complete aux-classpath) but are not analysed. Fail-safes unchanged: build/config changes and master/snapshot builds get a full reactor and full scan.Measurements
-pl)Local (clean tree, JDK 21, warm cache): full scan 75s vs 95s; scoped 23.5s vs 75s. Findings gate green and result-identical in every configuration. With both changes the spotbugs lane drops from pipeline critical path to the shortest Maven lane for typical PRs.
🤖 Generated with Claude Code