Test what both front ends actually draw - #17
Merged
Merged
Conversation
The unit tests run against a simulated DOM. It has no layout engine and no stylesheets, so it can say a node exists but never that it is visible, the right size, or a readable colour against what is behind it. Every visual bug this project has shipped was of exactly that kind, and every one reached a person before it reached a test. Eight Playwright tests against the published page: the demo replaces the clip, the tree is drawn inside its panel, every step-log line is readable against its background, the shape list fits what it holds, each shape draws the edges it claims, choosing a subgraph lights it up in the input graph, play advances the search, and nothing spills sideways at 420px wide. The difference is measurable. Reintroduce the dark-on-dark log text, or clip the shape list, and all 94 unit tests still pass while the browser tests fail. Both of those shipped. CI gains a browser job, and the Pages deploy runs them before publishing, so a demo that is broken in a browser cannot reach the site. Screenshot comparison is deliberately not used: fonts render differently on macOS and on the Linux runner, so pixel diffs would fail for reasons unrelated to the change.
The views had no automated coverage, and every visual bug this project shipped reached a person before it reached a test: dark text on a dark ground, a search opening too small to read, shapes drawing identically, a list cramped by a fixed height, a control that quietly stopped being built. What they had in common was not a mishandled click. It was the wrong nodes being built, or built with the wrong style. FxTest starts the toolkit once and builds real controls on the JavaFX thread with the stylesheets applied, without showing a window. No UI driving framework: TestFX is the obvious candidate and has had no release since 2023, against a JavaFX now three majors further on. - TreeRendererTest: a box for the root and each node reached, only what the search has reached, results marked found, dead branches marked dead, the asked-for subgraphs picked out, and exactly one state per box. - ReadableTest: every word in a panel differs in brightness from what is painted behind it, in both appearances. Removing the dark sheet's rule for the step log fails it, which is the bug that shipped. - EsuAppTest: the window offers every control, a pill per subgraph size, the menu items, and starts with stepping disabled. Dropping the save button, dropping a pill, or leaving controls live all fail. CI runs them under xvfb: JavaFX needs a display even when no window is shown.
Vitest's default pattern matches any .spec.ts, so it collected the Playwright tests and failed on them. It is now told to look only in test/, leaving browser/ to Playwright. The failure was on screen locally and I read past it: the summary line I checked said 94 tests passed, while the line above it said one test file had failed. CI, which checks the exit code, caught it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The views had no automated coverage, and every visual bug this project has
shipped reached a person before it reached a test: dark text on a dark ground,
a search opening too small to read, shapes drawing identically, a list cramped
by a fixed height, a control that quietly stopped being built.
What those had in common was not a mishandled click. It was the wrong nodes
being built, or built with the wrong style — which is what this can see.
The demo, in a real browser
The existing tests run against a simulated DOM. It has no layout engine and no
stylesheets, so it can say a node exists but never that it is visible, the
right size, or a readable colour against what is behind it.
Eight Playwright tests run the published page in Chromium: the demo replaces
the clip, the tree is drawn inside its panel, every step-log line is readable
against its background, the shape list fits what it holds, each shape draws the
edges it claims, choosing a subgraph lights it up in the input graph, play
advances the search, and nothing spills sideways at 420px wide.
The difference is measurable. Reintroduce the dark-on-dark log text, or clip
the shape list, and all 94 unit tests still pass while the browser tests
fail. Both of those shipped.
The desktop app, without a window
FxTeststarts the toolkit once and builds real controls on the JavaFX threadwith the stylesheets applied, showing nothing.
No UI-driving framework. TestFX is the obvious candidate and is the closest
thing JavaFX has to Playwright, but its last release was August 2023 against a
JavaFX now three majors behind ours — a dormant dependency is not a safety net.
Playwright itself cannot drive JavaFX at all; it automates browsers.
the search has reached, results marked found, dead branches marked dead, the
asked-for subgraphs picked out, exactly one state per box.
behind it, in both appearances. Asks of the resolved styles what ThemeTest
asks of the stylesheet text, so the two catch the same bug independently.
the menu items, and starts with stepping disabled.
Mutations, all killed: the save button dropped from the toolbar, a size pill
dropped, controls left live before a graph is loaded, and the dark sheet's rule
for the step log removed.
CI
A browser job, and
xvfbfor the Java tests — JavaFX needs a display even whenno window is shown. The Pages deploy runs the browser tests before publishing,
so a demo that is broken in a browser cannot reach the site.
What this still cannot do
Say whether the result looks good. Screenshot comparison is deliberately not
used: fonts render differently on macOS and on the Linux runner, so pixel diffs
would fail for reasons unrelated to the change. A screenshot on the pull
request remains the honest evidence for anything visual, and the templates now
say so.
79 Java tests, 94 unit, 8 browser.