Skip to content

Visual testing 2/8: specs testing visual command surface, manifest, and status #712

Description

@nathanacurtis

Parent: #588 — step 2 of 8.

Problem

There is no command. Everything visual testing does today runs from npm scripts in a separate test repo, against paths and ports that repo resolves itself, which is why no customer can run any of it.

This step lands the command surface plus the two stages that need neither a browser nor a network call — so the whole thing is testable before either dependency exists.

Potential solution(s)

  • specs testing visual registered the way specs storybook and specs bridge already are, with the testing namespace left open for the suites that exist today only as scripts (parity, emitted-tree typecheck, render round-trip, schema compliance). Universal flags: --config, --components, --kinds, --verbose.
  • specs testing visual init [--force] — scaffold testing/visual/package.json declaring Playwright, pixelmatch and pngjs, then print the one install command for the customer to run. The same contract as specs storybook init: we never ship, vendor, or install those packages, and the published CLI gains no dependency. --force rewrites the scaffold without touching anything hand-tuned beside it. Everything in this step works with nothing installed.
  • specs testing visual manifest [--check] — for every spec carrying a Figma source node id, locate the node in whichever fetched payload holds it, enumerate variant children, and precompute the story-side join inputs. --check is a dry run reporting unmapped Figma props and nothing else. Reads payloads through the sectioned-file reader rather than walking JSON by hand.
  • specs testing visual status [--json] — present or missing per shootable variant, and nothing more. No staleness model: missing is knowable with no model at all, and there is no trigger to compute because capture is overt.
  • State lands in the library's own testing/visual/ directory, not under storybook/ — that tree is generated wholesale and rewritten by init --force, so anything hand-tuned kept there is waiting to be overwritten.

Prop mapping stays declared, never guessed: a prop's Figma extension name, else its camelCase form when that key exists, else it lands in a reported unmapped list. Values coerce by the spec's declared type and then onto the emitted contract's enum casing, matched case-insensitively — the spec carries Figma's casing, the emitters normalize it, and stories and selectors all use the contract's. A prop present in the spec but absent from the contract does not exist at render time, so variants driving it are marked deferred rather than failed.

Compositions take the single-node branch: one node, one variant, empty config.

Acceptance criteria

  • manifest --check on the muse library reports zero unexplained unmapped props.
  • status on a library with no baselines reports every shootable variant missing.
  • Manifest output is byte-identical across two runs on unchanged input.
  • Each composition entry carries exactly one variant.
  • Nothing in this step opens a browser or makes a network call.
  • init writes the scaffold and prints an install command; it never runs an install itself.
  • init --force rewrites the scaffold and leaves a hand-tuned ignore file beside it untouched.
  • The published CLI's dependency list is unchanged by this step.

Case data

  • Territory: cli
  • Size: l

Notes

Depends on step 1.

Activity

  1. self-assigned this
    on Oct 8, 2026
  2. nathanacurtis commented on Oct 8, 2026

    @nathanacurtis
    MemberAuthor

    Built and verified on feature/testing-visual in specs (commit 0198a8e), on top of feature/compositions-cli. Validation evidence is in the commit message; #718 carries the remaining fixture sweep and docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

clispecs-cli commandstestingspecs-testing parity validation

Type

Fields

Priority

None yet

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions