You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BrowserTest: a skipped browser spec is reported as Passed #4590
wheels.wheelstest.BrowserTest.beforeAll() sets this.browserTestSkipped = true and returns quietly when:
the Playwright JARs are missing (Wheels.BrowserNotInstalled);
the JVM is unavailable;
the browser launch fails or times out;
CI is detected without opt-in.
browserDescribe()'s aroundEach then returns without calling the spec body. TestBox counts every such it as Passed, and the run reports totalSkipped: 0.
A spec whose only assertion can't pass (expect(...).toBe("probe")) reports 1 pass, 0 fail when Playwright is incomplete (for example, ~/.wheels/browser/lib holding playwright-*.jar and driver-*.jar but not driver-bundle, gson, Java-WebSocket or slf4j-*). Nothing in the output says the browser layer was unavailable. A local "green" browser run then proves nothing.
Expected
A skipped browser spec is reported as Skipped, with the reason.
Suggested fix (verified in an application subclass)
TestBox's skip(message) (BaseSpec.skip) marks the current spec Skipped. Registering a beforeEach inside browserDescribe() that calls it when the browser layer is unavailable gives 0 pass / 0 fail / N skipped, with no spec body run:
public void function browserDescribe(required string title, required any body) {
var me = this;
var innerBody = arguments.body;
describe(arguments.title, function() {
beforeEach(function() {
if (me.browserTestSkipped) skip(me.$browserSkipReason());
});
// ... existing beforeEach($startBrowserContext) / aroundEach / afterEach ...
innerBody();
});
}
beforeAll() can record the reason in each catch branch: the missing JAR from Wheels.BrowserNotInstalled's message, the launch error's message, or "CI without WHEELS_BROWSER_CI_ENABLE".
Note: TestBox's skipargument to it() / describe() doesn't work for this. A closure there is evaluated when the suite is registered, before beforeAll() has decided whether to skip.
Environment: Wheels 4.0.x, Lucee 7.0.4, TestBox as bundled in vendor/wheels/wheelstest/system.
This is fixed on develop by #4263 (merged 2026-10-04), so it's in 4.2.0. The report was against 4.0.x.
browserDescribe() now calls TestBox's skip() with a reason whenever the browser layer is unavailable, so those specs are reported as Skipped, not Passed. Verified on Lucee 7 + SQLite at develop 432ae4ec6, running directory=wheels.tests.specs.wheelstest:
Playwright not installed: pass 300 / fail 0 / error 0, totalSkipped 32. Every browserDescribe spec has status Skipped with Playwright is not installed. Run \wheels browser setup` to run browser specs.` Those bundles report pass 0.
WHEELS_CI=true without WHEELS_BROWSER_CI_ENABLE: the same 32 specs are Skipped, with Browser specs are skipped in CI (WHEELS_CI is set). Set WHEELS_BROWSER_CI_ENABLE=true to run them.
Upgrading to 4.2.0 resolves it. Closing; please reopen if you still see a browser spec reported as Passed on 4.2.x.
Activity
bpamiri commentedon Oct 15, 2026
This is fixed on
developby #4263 (merged 2026-10-04), so it's in 4.2.0. The report was against 4.0.x.browserDescribe()now calls TestBox'sskip()with a reason whenever the browser layer is unavailable, so those specs are reported as Skipped, not Passed. Verified on Lucee 7 + SQLite at develop432ae4ec6, runningdirectory=wheels.tests.specs.wheelstest:totalSkipped32. EverybrowserDescribespec has status Skipped withPlaywright is not installed. Run \wheels browser setup` to run browser specs.` Those bundles report pass 0.WHEELS_CI=truewithoutWHEELS_BROWSER_CI_ENABLE: the same 32 specs are Skipped, withBrowser specs are skipped in CI (WHEELS_CI is set). Set WHEELS_BROWSER_CI_ENABLE=true to run them.Upgrading to 4.2.0 resolves it. Closing; please reopen if you still see a browser spec reported as Passed on 4.2.x.