Skip to content

Fix exact visual thresholds and expand automation coverage - #88

Merged
AdamXweb merged 2 commits into
mainfrom
fix/automation-test-coverage
Oct 30, 2026
Merged

AdamXweb merged 2 commits into
mainfrom
fix/automation-test-coverage

Conversation

@adamXbot

@adamXbot adamXbot commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Close the automation gaps found while reviewing #85. A changed pixel on a large capture could round to 0.0000% and pass a zero-change limit; checks now compare exact pixel counts. Generated previews now run for dependency changes anywhere in a theme repository.

  • Compare all three capture pairs (check against both shot runs, plus shot against shot), report pixel counts, and upload original screenshots on failure. The checks use the shared comparison noise rules without adding a percentage allowance.
  • Share capture, sensitivity and state/surface checks between PR CI and scheduled Stable/ESR/Beta/Nightly jobs. Preserve the narrowly defined search-button exception.
  • Test the advertised Pillow 10.1.0 minimum in a full Linux/Python 3.12 unit job.
  • Extend Firefox smoke fixtures for optional-sheet imports and decoded assets, shipped customChrome.css in preview/watch mode, and cleanup of all recorded Windows process IDs.
  • Record tab state on fixture-navigation timeouts. Use bounded recovery only when that state shows no attached content process.
  • Capture modal dialogs through Firefox's regular snapshot renderer, awaiting settled frames at native resolution and releasing each bitmap.
  • Exercise release orchestration offline and document variant-name migration when optional-sheet names collide.

Validation

  • Python 3.14 / Pillow 13: 593 tests run, one skipped, no failures or errors.
  • Python 3.12 / Pillow 10.1.0: 40 focused comparison, threshold, dialog and capture-state tests passed.
  • Python 3.9 grammar checks, workflow YAML parsing and whitespace checks passed.
  • Windows, macOS, Linux and Linux-ESR browser checks, unit jobs and installed-package jobs passed on e05e200. The replay onto current main preserves that source tree exactly; GitHub will rerun checks for the new commit IDs.

Based on current main, which includes #85. No merge or release is performed by this PR.

@adamXbot
adamXbot force-pushed the fix/automation-test-coverage branch from 0a462e9 to f21bf76 Compare October 30, 2026 05:25
@adamXbot
adamXbot force-pushed the fix/review-round-two branch from 1ebe0c9 to 4e3ffba Compare October 30, 2026 05:25
@adamXbot
adamXbot force-pushed the fix/automation-test-coverage branch 2 times, most recently from c087631 to e05e200 Compare October 30, 2026 06:11
Base automatically changed from fix/review-round-two to main October 30, 2026 10:30
Replay the automation coverage changes on the updated review branch, retaining its targeted antialiasing filter and default canvas settings.
Use currentWindowGlobal.drawSnapshot at native resolution and await settled frames. Release every bitmap, including on draw failures. Keep the default canvas context restored by the base branch; do not reintroduce its withdrawn software-canvas workaround.

Resolves the overlapping dialog capture edits with fix/review-round-two while retaining the base comparison changes.

Validation: 593 tests passed, one skipped; 40 focused checks passed with Pillow 10.1.0. The single-pixel and thin-border regression tests still pass. Live rendering requires CI confirmation.
@adamXbot
adamXbot force-pushed the fix/automation-test-coverage branch from e05e200 to 37eb2a4 Compare October 30, 2026 10:37
@AdamXweb
AdamXweb merged commit 9161e87 into main Oct 30, 2026
19 checks passed
@AdamXweb
AdamXweb deleted the fix/automation-test-coverage branch October 30, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants