Skip to content

Feedback from a breakpoint parity run: side labels, publishing zero-diff shots, noisy varies output #113

Open

Description

@tmashara

Status (2026-11-02)

  • Done: varies count labelled "painted out", mask suggestions skip bare-tag selectors, --rerun-failed lists only the rerun's findings (#114); --only <route|file> runs a config subset with its hooks (#118).
  • Moved to Publish a report: --base-url for the markdown export and renderdiff publish #104: zero-diff dedupe and WebP/tiling on publish.
  • Open: side labels (labels / --labels), skip mask suggestions when the layout moved, collapse findings for a hidden subtree.

From a 140-shot breakpoint run on 0.12.0 (base build vs change build, 20 routes × 7 widths) and earlier runs on the same site.

  • Name the sides. left/right in report.html (figcaptions, slider label, crop titles, summary) doesn't say which build is which, and --markdown says before/after instead. Add labels: { left: "current", right: "prod" } to the config (or --labels current,prod) and use it everywhere: figcaptions, the meta line, region titles, bands, the slider's aria-label and the markdown export. The right label depends on the reference, e.g. master for a local build of the base branch, so it must not be hard-coded.
  • Zero-diff shots when publishing (Publish a report: --base-url for the markdown export and renderdiff publish #104). A shot with 0 differing pixels only needs one image. Our local compaction script stored the left image once and dropped the right and diff ones, then crashed: the slider's <img src="….right.png"> had no file left. If publish dedupes, the slider and the diff figure need to point at the shared image or a "no differing pixels" note.
  • WebP and tiling on publish (Publish a report: --base-url for the markdown export and renderdiff publish #104). Reports are ~2.5 GB of PNG on a 200-route run, too big for GitHub Pages. Converting to WebP and cutting shots taller than 16,383 px (WebP's limit) into tiles brought it to ~500 MB.
  • varies totals don't match their regions. /case-studies/alto@1240 printed "varies 48,263px" with a mask suggestion, but the region was 1 px tall and the shot had 0 diff px.
  • Mask suggestions are fragile. On an intended layout change, varies suggested masking p:nth-of-type(14) and four other bare paragraph selectors (1.5M px). Those would also hide real changes. Suggest only selectors with an id or class, and don't suggest for shots that differ on every load anyway because the layout moved.
  • --rerun-failed reprints the old run's blocks. The summary still shows the first run's varies and unmatched sections, so you can't tell what the rerun found.
  • Collapse text findings for a hidden subtree. When the nav switches to a hamburger, every menu item is reported as removed text (-"Administration" ×20 routes, ~100 lines), which pushes the summary off-screen. Report "nav subtree hidden" once instead.
  • Run a config's hooks on a subset. Routes given on the CLI lose their config hooks, so a subset run means copying the config next to a trimmed routes.txt. A --only <route|file> filter over the config's routes would keep hooks and masks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions