Skip to content

optimise build pipeline around webgpu design, removing legacy react constraints - ensure full test criteria remains #588

Open

Description

@MaximumTrainer

Context

Now that next/svelte-webgpu is the primary target and the React/three.js tree is being retired under #542 (AC1 = PR #587, 311 files / ~49k lines deleted), the build + test pipeline still carries React-era deps, exclusions, and gates, and has thin graphics-testing beyond frozen pixel baselines. This issue scopes the clean-up and the WebGPU-first additions (mutation testing, deeper graphics tests, broader OS targets) so the pipeline matches the shipped stack.

Sequencing: this follows #587 (React removal), overlaps with #542 AC12 cutover and #548 (GPU runner, dormant). Backlog until Dan says go.

Grounding (branch next/svelte-webgpu, HEAD fc4eb61): package.json, vite.config.ts, vitest.config.ts, playwright/playwright.config*.ts, .github/workflows/next-branch-ci.yml, .github/workflows/playwright-e2e-clean.yml, src/lib/renderer/passes/__tests__/wgsl.test.ts, src/__tests__/shaders.test.ts, playwright/tests/helpers/assertCanvasNotBlank.ts.


Functional Requirements

FR1. Drop React/three.js deps from package.json.
Runtime: react, react-dom, @react-three/{drei,fiber,postprocessing}, react-chartjs-2, postprocessing, three (lines 42–53). Dev: @vitejs/plugin-react, @testing-library/react, @react-three/test-renderer, eslint-plugin-react-hooks, eslint-plugin-react-refresh, @types/react, @types/react-dom, @types/three (lines 68–85).

FR2. Prune coverage config.
Remove vitest.config.ts coverage.exclude entries that resolve to deleted React files (lines 42–118) and per-file ratchets at lines 207–241 (landscapeLayout.ts, foliage*, boatComponents.tsx, skyComponents.tsx, wakeTexture.ts, …). Re-measure global thresholds against the surviving src/lib/** denominator and re-pin per-file floors.

FR3. Retire legacy GLSL shader tests.
Remove src/__tests__/shaders.test.ts and the @shaderfrog/glsl-parser dep. Any invariant worth keeping (shaderChunks/shaderErrors) migrates to WGSL equivalents.

FR4. Mutation testing.
Introduce a mutation-testing harness scoped to critical engine logic, so tests that pass without exercising behaviour are flagged. No current scaffold anywhere on the branch.

FR5. WGSL contract tests beyond parse.
Expand src/lib/renderer/passes/__tests__/wgsl.test.ts from "has ≥1 entry point" (its current shape) to assert:

  • binding-group layouts per pass against a committed snapshot,
  • uniform-struct sizes per pass against a committed table — the Frame struct's 272 B / 68-float layout (#542 P1 — UI shell in Svelte (UI track) #549 regression) is one row.

FR6. Compute-pass unit tests.
Cover every compute pipeline under src/lib/renderer/plan/* (e.g. the WebGPU culler) with input→output assertions that don't need a visible canvas.

FR7. Per-region pixel probe.
Generalise playwright/tests/helpers/assertCanvasNotBlank.ts (20×20 grid, 20% non-blank) into a samplePerRegion({sky, water, boat, …}) variant so a scene filled with sky can't pass as "not blank".

FR8. OS target coverage for the engine.
.github/workflows/next-branch-ci.yml runs ubuntu-only (preflight, smoke, e2e-shards, webgpu-engine) with an optional GPU_RUNNER_LABEL leg. Expand SwiftShader-safe jobs beyond ubuntu, and declare explicit coverage for the three WebGPU backends: Linux Vulkan, macOS Metal, Windows D3D12.

FR9. Engine visual baselines on real GPU.
SwiftShader has no WebGPU backend, so engine baselines must come from a GPU runner. playwright.config.visual.ts's "a baseline is a picture of one renderer" rule still holds — pick one platform and record there.

FR10. Perf-budget telemetry on real GPU.
Re-home the behaviour of route-performance-budgets.spec.ts (currently in playwright.config.next.ts's ignore list) onto the GPU runner using the ride harness's __FRAME_STATS.gpuMs/cpuMs and __RENDER_STATS.drawCalls.

FR11. Prune legacy CI.
.github/workflows/playwright-e2e-clean.yml jobs whose only purpose is the deleted React tree (incl. legacy webgpu-spike) are retired or re-pointed at the new engine's specs after AC12 cutover.


Non-Functional Requirements

  • NFR1. PR median wall-clock does not regress by >50%; scheduled jobs may be longer.
  • NFR2. No test file is deleted without (a) another test owning its invariant or (b) an in-issue note retiring it.
  • NFR3. Mutation testing is time- or scope-capped so it never blocks the critical path.
  • NFR4. Coverage thresholds never ratchet down silently (project rule); a lowering carries a vitest.config.ts comment with reason (VR-23: Make the R3F scene unit-testable: plan functions and @react-three/test-renderer smoke tests #343 pattern).
  • NFR5. Any job that asserts engine behaviour needing a real adapter declares that via GPU_RUNNER_LABEL; it must never silently pass by falling back to SwiftShader.
  • NFR6. New scheduled workflows skip-with-warning when a required secret is absent (project convention).

Acceptance Criteria

  • AC1. package.json carries none of the FR1 deps; npm ci && npm run build && npm run verify passes; rg "from 'react'" src/ returns zero hits.
  • AC2. vitest.config.ts has no coverage.exclude glob that resolves to zero files; thresholds re-pinned numerically against the lib-only tree; per-file ratchets refer only to extant files.
  • AC3. src/__tests__/shaders.test.ts gone, @shaderfrog/glsl-parser removed from package.json.
  • AC4. npm run test:mutation wired; a nightly/weekly workflow runs it (skip-with-warning), uploads the HTML report, and records a baseline survived-mutant count in CLAUDE.md. At least one assertion is added as a direct consequence of a surviving mutant.
  • AC5. wgsl.test.ts asserts entry points and per-pass binding-group layout snapshot and uniform-struct sizes; the 272 B / 68-float Frame row is explicit; a seeded 256 B regression (replica of #542 P1 — UI shell in Svelte (UI track) #549) flips the test red.
  • AC6. Every compute pipeline in src/lib/renderer/plan/* has an input→output test; a seeded off-by-one in the culler flips a test red.
  • AC7. assertCanvasNotBlank.samplePerRegion exists and is used by at least webgpu-engine-smoke.spec.ts; a seeded "clear-to-sky" scene fails it.
  • AC8. next-branch-ci.yml runs its SwiftShader-safe specs on at least [ubuntu, windows] (macOS per D6) and declares its engine jobs on the chosen GPU runner; representative PR green.
  • AC9. Visual baselines for the engine are recorded on the chosen GPU-runner platform (D8); a visual job on next/svelte-webgpu passes PR runs after a baseline-affecting change behind the visual-baseline label.
  • AC10. A perf-budget spec on the GPU runner flips red when the Henley short route at tier medium regresses __FRAME_STATS.gpuMs by the D9 threshold.
  • AC11. No .github/workflows/* job whose only purpose is the deleted React tree remains; CLAUDE.md Commands table updated.
  • AC12. PR median wall-clock before/after is captured in the implementation PR body; stays within NFR1.

Decision points (Dan to answer tersely)

D1. React dep removal timing (FR1).
a. Fold into #587 as one atomic cutover.
b. Immediate follow-up PR after #587 merges (already on the memory's "next" list).
c. Bundle with D2 coverage-config cleanup.

D2. Coverage re-base strategy (FR2).
a. Measure post-#587, pin measured numbers with a comment per the existing vitest.config.ts:194-197 pattern.
b. Add per-file 100% ratchets for every surviving src/lib/** file already ≥95%.
c. Keep the global floor only; retire all per-file ratchets.

D3. Mutation testing framework & scope (FR4).
a. StrykerJS + vitest runner, scope src/lib/renderer/plan/** + physics modules.
b. StrykerJS, scope encoders/parsers only (FIT encoder, intervals.icu service, utils).
c. Defer to a separate issue after AC12 cutover.

D4. WGSL layout snapshotting (FR5/AC5).
a. Hand-written contract per pass using wgsl_reflect.
b. Inline snapshot (toMatchInlineSnapshot) extending the existing wgsl.test.ts.
c. External passes/<name>.layout.json schema file per pass.

D5. Compute-pass test host (FR6/AC6).
a. Node + @webgpu/node polyfill under vitest.
b. Only on the GPU runner, as a dedicated Playwright page that reads back a buffer.
c. Both — polyfill unit + GPU-runner end-to-end.

D6. OS targets for next-branch-ci.yml (FR8/AC8).
a. [ubuntu, windows, macos] for SwiftShader-safe jobs; engine on GPU_RUNNER_LABEL.
b. [ubuntu, windows] only (mirror legacy webgpu-spike).
c. Keep ubuntu-only; spend the budget on GPU-runner matrix instead.

D7. GPU-runner matrix (FR8).
a. One Linux GPU runner (current GPU_RUNNER_LABEL); Metal/D3D12 untested on hosted side.
b. Add a Windows GPU runner (D3D12) when available.
c. Add Windows + macOS self-hosted GPU runners (biggest coverage, biggest ops cost).

D8. Engine visual-baseline platform (FR9/AC9).
a. Linux GPU runner (continuity with existing single-platform rule).
b. Windows GPU runner (closest to Dan's dev device per memory).
c. macOS GPU runner.

D9. Perf-budget source (FR10/AC10).
a. Delta vs previous run (committed baseline JSON updated by a label).
b. Absolute budget per tier (basic≤5ms, medium≤10ms, high≤16ms gpuMs).
c. Both — absolute cap + regression delta.

D10. Mutation + full-matrix scheduling (NFR3).
a. Scheduled nightly on next/svelte-webgpu and main.
b. Weekly on both.
c. Per-PR behind a label (mutation, full-matrix).

D11. Legacy pipeline retirement (FR11/AC11).
a. Delete playwright-e2e-clean.yml + webgpu-spike once AC12 cutover completes.
b. Keep them against the legacy React path on the pre-cutover tag for 30 days, then delete.
c. Delete at this issue's merge regardless of cutover state.


Test scenarios

  • TS1 (AC1). Clean checkout → npm ci && npm run build && npm run verify → green; rg "react" package.json returns no runtime hits.
  • TS2 (AC2). CI step walks coverage.exclude globs and fails if any resolves to zero files; thresholds are numeric literals.
  • TS3 (AC3). rg "@shaderfrog/glsl-parser" . returns zero; src/__tests__/shaders.test.ts absent.
  • TS4 (AC4). npm run test:mutation produces a report under the time cap; CI uploads the HTML artifact.
  • TS5 (AC5). Replica of #542 P1 — UI shell in Svelte (UI track) #549: shrink Frame from 272 B → 256 B in the WGSL, test flips red before any pixel renders.
  • TS6 (AC6). Seeded instance buffers in a createWebGPUCuller test: in-frustum, behind-camera, past-cull-distance — expected visible count hand-computed.
  • TS7 (AC7). samplePerRegion({sky, water, boat}) on a scene seeded clear-to-sky — fails because boat region is pure sky.
  • TS8 (AC8). Representative PR green across the new OS matrix; a seeded path-separator bug in a config loader is caught by the Windows leg.
  • TS9 (AC9). 1-px tint change in a fragment shader flips the engine visual job red; visual-baseline label re-records and the committed PNG matches the chosen GPU runner.
  • TS10 (AC10). 25% gpuMs regression on Henley short / medium flips the budget spec red; 5% stays green.
  • TS11 (AC11). .github/workflows/* contains no job gated on React/legacy; CLAUDE.md Commands table reflects reality.
  • TS12 (AC12). PR body carries wall-clock-before.txt + wall-clock-after.txt from the implementation PR's own CI runs; delta ≤ NFR1 bound.

Overlap / conflicts

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions