Skip to content

ci: CPD without compile + changed-modules-only lint lane#1457

Draft
joaodinissf wants to merge 6 commits into
dsldevkit:masterfrom
joaodinissf:perf/lint-lane-step-5
Draft

ci: CPD without compile + changed-modules-only lint lane#1457
joaodinissf wants to merge 6 commits into
dsldevkit:masterfrom
joaodinissf:perf/lint-lane-step-5

Conversation

@joaodinissf

@joaodinissf joaodinissf commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Note

Stacked on #1456. Earlier commits belong to the perf stack; review only the last two commits here.

Two reductions to the lint lane (~3.5 min in CI):

  1. CPD runs without a compile pass — CPD tokenizes sources under src/ and needs neither bytecode nor the target platform. cpd.xml outputs verified identical with and without compile, at both the current token threshold and the PMD default of 100 (so the result holds after chore: enable CPD detection (pmd.cpd.min → 100), de-dup and baseline #1397). Locally 6.8s vs 44.3s; the CI step was 53s of which ~40s was redundant recompilation.
  2. Selective lint — compute-spotbugs-skip.sh becomes compute-analysis-skip.sh <base-sha> <spotbugs|lint>: the lint mode injects pmd.skip/cpd.skip/checkstyle.skip into unchanged modules' poms and exports LINT_SCOPE_ARGS (-pl ../ddk-target,<changed> -am), mirroring the spotbugs lane (including the ddk-target reactor pin and the report-presence gate from ci: in-process SpotBugs + changed-modules-only reactor in the spotbugs lane #1456; only source-bearing bundles are expected to report, so a source-less branding bundle can never false-fail the gate). compile stays in the PMD/Checkstyle invocation (PMD's type-resolving rules need Tycho's aux-classpath; the -am-pulled unchanged dependencies compile but are not analysed). When no reactor module changed at all (e.g. a docs-only PR) both lint invocations and the SARIF upload are skipped entirely (LINT_KEPT=0) — the upload guard mirrors the spotbugs lane's, since upload-sarif fails on an empty merge directory. Fail-safes unchanged (the full-scan path exports KEPT=all so the workflow guard can never mistake it for the empty case), and changes under ddk-configuration/ (rulesets, filters) now trigger the full-scan path in both lanes.

Measurements

Regime Before After Δ
Full-scan regime, warm cache (this PR's runs vs the lint references) 199–219s 139–172s across runs −14–37% (CPD-without-compile)
Scoped run, one changed module, cold cache (fork probes vs same-conditions full lint) 288s 96–132s across runs (final-SHA runs 96–103s) −54–67%
Docs-only PR, no reactor module changed (fork probe) 288s (full lint) 10–13s −95%

Local (one changed module): PMD+Checkstyle invocation 11.7s vs 33.8s full, CPD 3.9s vs 20.1s, only the changed module reports, zero violations; CPD outputs identical at both token thresholds.

Code Scanning semantics: repo-wide alert state reflects the default branch, which receives no lint/spotbugs analyses (verify uploads on PR refs only), so a scoped upload that omits unchanged modules affects PR-context annotations only — the same property the spotbugs category has had since the per-module skip landed.

Merge chain: #1399 (merged) → #1396 → #1400 → #1456 → #1457 (this PR) → #1397

🤖 Generated with Claude Code

@joaodinissf joaodinissf mentioned this pull request Jul 10, 2026
Closed
@joaodinissf joaodinissf force-pushed the perf/lint-lane-step-5 branch from 3ecbf45 to e6a7274 Compare July 10, 2026 22:10
@joaodinissf joaodinissf mentioned this pull request Jul 10, 2026
Closed
@joaodinissf joaodinissf force-pushed the perf/lint-lane-step-5 branch 2 times, most recently from b78df4f to c7f7b05 Compare July 11, 2026 14:25
This was referenced Jul 11, 2026
Closed
Closed
Closed
Draft
Draft
Draft
Open
@joaodinissf joaodinissf force-pushed the perf/lint-lane-step-5 branch 2 times, most recently from bbf2dde to 063479f Compare July 11, 2026 21:08
joaodinissf and others added 3 commits July 11, 2026 23:13
@joaodinissf @claude
ci: fast static analysis with early-fail lint + inline SARIF
91239aa 
Replace the pmd/checkstyle jobs with a parallel shape that gives early,
inline feedback and stops re-running analysis inside the build:

  - lint: compile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check,
    -T 2C, --fail-never; gates by counting the merged SARIF (+ cpd.xml grep).
    Fails in ~3-5 min on its own check, independent of the build.
  - spotbugs: compile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel lane
    (the slow analysis).
  - maven-verify: build + tests only; the redundant checkstyle/pmd/spotbugs
    goals are dropped (now owned by lint/spotbugs).
  - line-endings: unchanged.
  - both new lanes restore the master-produced Linux-maven-publish-* cache,
    restore-only, mirroring snapshot.yml's path spec and key recipe exactly
    (the path spec is hashed into the cache version, so the mirroring must
    be literal).

All three emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning
(security-events: write) for inline annotations on the PR diff + Security tab.
No custom Python annotator.

Count-gate rather than the *:check goals: the check goals @Execute-fork a
second analysis and cannot emit SARIF, and without the full compile classpath
they false-positive on type-resolving rules. Each report goal runs once
(full-reactor compile -> correct + SARIF) and the gate counts the result.
Rationale + tables in docs/ci-static-analysis-design.md; measurement protocol
in docs/ci-measurement-protocol.md.

CPD gating is wired but inert until dsldevkit#1339 lowers the token threshold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joaodinissf @claude
ci: scope SpotBugs to the PR's changed modules (per-module skip)
8ce709a 
SpotBugs' per-module analysis is the spotbugs job's long pole. A PR only needs
its changed modules scanned, so a pre-step injects <spotbugs.skip>true> into
every unchanged reactor module's pom — the plugin then skips the goal, and the
per-module JVM fork, for them. The full-reactor compile is kept (a changed
module keeps its complete aux-classpath); a build/config change falls back to a
full scan. pull_request only — master/snapshot run a full scan.

-Dspotbugs.onlyAnalyze was the cleaner-looking alternative but screens too late
(after the per-module fork), ~17% vs ~88% measured; the script header documents
the migration if an upstream SpotBugs early-exit ever lands.

- .github/scripts/compute-spotbugs-skip.sh: diff -> changed modules -> inject
  skip into the unchanged ones (idempotent; build/config change -> full scan).
- verify.yml spotbugs job: fetch-depth 0 + a scope step before compile;
  -Djgit.dirtyWorkingTree=ignore because the scope step dirties poms on purpose
  and this job releases nothing (releases/maven-verify keep =error); SARIF upload
  guarded so an empty scan set (no module scanned) doesn't fail the upload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joaodinissf @claude
ci: run SpotBugs in-process in the verify spotbugs lane
e187c63 
spotbugs-maven-plugin forks a fresh JVM per module by default (maxHeap
2048), so a full scan pays 59 JVM startups. -Dspotbugs.fork=false runs
the analysis inside the Maven JVM; the lane's MAVEN_OPTS -Xmx4g governs
the shared heap (the plugin's maxHeap setting applies to forks only).

Measured locally (full scan, clean tree, JDK 21): 75s vs 95s wall at
-T 3C, and stable at CI thread width (-T 8, 4 GB heap) with identical
findings (59 module SARIFs, same result count).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joaodinissf joaodinissf force-pushed the perf/lint-lane-step-5 branch 2 times, most recently from c3fcc98 to 0f1f731 Compare July 11, 2026 21:44
joaodinissf and others added 3 commits July 11, 2026 23:53
@joaodinissf @claude
ci: build only changed modules + upstream deps in the spotbugs lane
6fa2d0e 
compute-spotbugs-skip.sh now also exports SPOTBUGS_SCOPE_ARGS
("-pl ../ddk-target,<changed modules> -am") and the spotbugs lane
passes it to mvn, so a scoped run builds only the PR's changed modules
plus their upstream dependencies instead of the full 64-module
reactor. The -am-pulled unchanged dependencies keep the injected
spotbugs.skip: they compile (complete aux-classpath) but are not
analysed.

ddk-target is pinned into every scoped reactor: the target-definition
artifact is referenced by target-platform-configuration, not by any
bundle MANIFEST, so -am alone never pulls it — Tycho then falls back
to a local-repository copy of the .target, which fails on a cold cache
and can silently resolve a stale target definition on a warm one
(verified: a June install of the same sequenceNumber still pointed at
the 2026-03 release train while the tree's points at 2026-06).

The gate verifies that every scanned source-bearing module produced
its SARIF (and that a full scan produced any at all): --fail-never
swallows even target-resolution failures, so without a presence check
a dead build uploads nothing, counts zero violations, and passes
vacuously.

Fail-safes are unchanged: a build/config change means a full reactor
and full scan, and a PR touching no reactor module builds the full
reactor with every analysis skipped.

Measured locally (single-module change, clean tree, JDK 21): 34s wall
vs 75s for the full-reactor equivalent, resolving the current target
platform (jdt.core 3.46.0) with findings identical to the full scan.
Verified against a repository with no installed ddk-target: without
the pin the run reproduces the swallowed resolution failure with zero
SARIFs; with it the scoped run succeeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joaodinissf @claude
ci: run CPD without a compile pass in the lint lane
520bf95 
CPD tokenizes sources under src/ and needs neither bytecode nor a
resolved target platform, so the second lint invocation drops its
compile goals. cpd.xml outputs are identical with and without the
compile pass, verified at the current token threshold and at the
PMD default of 100 (timestamp attributes aside).

Measured locally (warm tree, JDK 21): 6.8s vs 44.3s at the current
threshold; 4.5s vs 28.5s at threshold 100. In CI the invocation was
53s, ~40s of it redundant recompilation and JVM startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joaodinissf @claude
ci: scope PMD/CPD/Checkstyle to changed modules in the lint lane
8161060 
compute-spotbugs-skip.sh becomes compute-analysis-skip.sh with a mode
argument: `spotbugs` injects spotbugs.skip as before, `lint` injects
pmd.skip, cpd.skip and checkstyle.skip, and each mode exports its
-pl/-am reactor scope args. The lint lane gains the scope step and
passes LINT_SCOPE_ARGS to both invocations; `compile` stays in the
PMD/Checkstyle invocation because PMD's type-resolving rules need
Tycho's aux-classpath (skip-injected -am dependencies compile but are
not analysed). Changes under ddk-configuration (rulesets, filters) now
also trigger the full-scan fail-safe in both lanes.

Code Scanning note: repo-wide alert state reflects the default branch,
which receives no lint/spotbugs analyses (verify runs on pull_request
only), so a scoped upload that omits unchanged modules can only affect
PR-context annotations — the same property the spotbugs category has
had since the per-module skip landed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joaodinissf joaodinissf force-pushed the perf/lint-lane-step-5 branch from 0f1f731 to 8161060 Compare July 11, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@joaodinissf