ui-smoke-tests flaky on samples/workflow: HealthService reports "OpenIDM failure during startup" 2 s after activation #217

Summary

The ui-smoke-tests job fails intermittently on the samples/workflow matrix legs with:

SEVERE: OpenIDM failure during startup, ACTIVE_NOT_READY:
        Required services not all started [org.openidentityplatform.openidm.api-servlet, ...]

The message is written before OpenIDM has finished coming up — INFO: OpenIDM ready follows within the same second — so the server is healthy by the time the Playwright tests would run. But the Start OpenIDM step greps the log for ERROR|SEVERE|Exception|Throwable (.github/workflows/build.yml:190-212, and again in Print openidm logs, :251-270), so the job goes red on a message that describes a state the server already left.

Root cause

HealthService schedules its startup readiness check with two different delays:

• HealthService.java:341 — scheduleCheckStartup(serviceStartMax), where serviceStartMax defaults to 15000 ms (HealthService.java:249, overridable via openidm.healthservice.servicestartmax). This path is taken when FrameworkEvent.STARTED arrives while the component is already active.
• HealthService.java:398 — scheduleCheckStartup(2000), taken when the OSGi framework is already started at the moment HealthService activates.

In the CI deployments it is the second path that fires, so the check runs 2 seconds after activation instead of 15. The 2000 is a hard-coded literal and does not honour openidm.healthservice.servicestartmax.

Measured gap between OpenIDM Health Service component is activated and the SEVERE line, across four independent failing jobs — exactly 2 s every time:

samples/workflow boots the Activiti engine and is the heaviest of the sample profiles, so api-servlet regularly needs more than 2 s to register — and registers a fraction of a second later, after which the state moves to ACTIVE_READY and INFO: OpenIDM ready is logged.

Log excerpt (PR #215), showing the SEVERE and the ready transition in the same second:

Sep 18, 2026 6:02:27 PM org.forgerock.openidm.info.impl.HealthService activate
INFO: OpenIDM Health Service component is activated.
...
Sep 18, 2026 6:02:29 PM org.forgerock.openidm.info.impl.HealthService$4 run
SEVERE: OpenIDM failure during startup, ACTIVE_NOT_READY: Required services not all started [org.openidentityplatform.openidm.api-servlet]
Sep 18, 2026 6:02:29 PM org.forgerock.openidm.servletregistration.impl.ServletRegistrationSingleton register...
...
Sep 18, 2026 6:02:29 PM org.forgerock.openidm.info.impl.HealthService setState
INFO: OpenIDM ready

Impact

This is the largest single source of red CI right now — it accounts for 7 of the 9 currently failing checks across the open PRs:

• Fix partial path traversal in UI ResourceServlet #202 — run 35360285248
• Fix query-filter escaping and HTML injection in UI filter editors #204 — run 35349909580
• Fix CodeQL error-level findings: dead branches, log arguments, locking, clone #205 — run 35335942650 (two legs)
• Fix CodeQL warning-level Java correctness findings #207 — run 35338910898
• Replace deprecated API calls flagged by CodeQL #213 — run 35354860456
• Add missing @Override annotations and exclude the unused-parameter query #215 — run 35358828158

That it is a pure timing flake is confirmed directly: branch fix-codeql-medium-alerts (#206) failed on this exact leg in run 35336946012 and then passed, unchanged in that area, in run 35365295299.

Beyond CI, a spurious SEVERE ... failure during startup in a normal server log is misleading for operators, who have no way to tell it apart from a real startup failure.

Suggested fix

Preferably fix the product side rather than the log filter:

1. Make the already-started path in HealthService.java:398 use serviceStartMax (or at least a configurable value) instead of the hard-coded 2000, so a single property governs both paths.
2. Optionally, have the check re-evaluate once instead of reporting on the first miss, or downgrade to WARN and only escalate to ERROR if the state is still not ACTIVE_READY after serviceStartMax.

If a workflow-side mitigation is wanted in the meantime, the Start OpenIDM step could ignore an ACTIVE_NOT_READY line that is followed by OpenIDM ready — but that hides the message rather than removing the cause.

Environment

• Branch: master (7.1.3-SNAPSHOT)
• Runner: ubuntu-latest, JDK 17 and 26, both context_path variants
• Sample: samples/workflow