Repository navigation
Leave DBOS unlaunched when launch() fails - #662
Merged
Merged
Conversation
launch() published its executor before running migrations and starting it. When either threw, the instance kept that executor: a second launch() returned at once with nothing running, and other calls failed with a NullPointerException on what it never started. Every deployment that runs with migrations off against a schema below the required version takes that path. Now a launch that throws shuts down whatever it started, clears the executor and rethrows, so the instance is as if launch() had never been called and a later launch() tries again from the start. Closing an executor that failed before creating its thread pools no longer throws.
… of it Catch Throwable rather than RuntimeException and Error, and clear the executor in a finally, so a checked exception thrown undeclared by a listener or Conductor provider, or an Error from close(), can't leave the half-started executor in place. Document that dbosShutDown can come without dbosLaunched when launch fails, and give the test that migrates a schema from empty the five-minute timeout CustomSchemaTest uses on slow CockroachDB runners.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Failed-launch cleanup can leave resources running or retain a stale executor reference when shutdown throws, allowing unsafe retries or blocking subsequent launches.
2 open issues
What changed in this PR
This PR improves DBOS startup recovery so the same instance can retry launch() after a failed attempt rather than retaining an unusable executor.
Changes:
- Adds failed-launch cleanup and executor reference reset handling.
- Guards executor shutdown against uninitialized thread pools and schedulers.
- Documents retry behavior and adds regression tests for schema validation and lifecycle listener failures.
| File | Description |
|---|---|
transact/src/test/java/dev/dbos/transact/FailedLaunchTest.java |
Tests failed-launch cleanup, unlaunched state, and successful retries. |
transact/src/main/java/dev/dbos/transact/execution/DBOSExecutor.java |
Adds null checks for services that startup may not initialize. |
transact/src/main/java/dev/dbos/transact/DBOS.java |
Handles launch failures and documents the retry contract. |
Review effort: Balanced
DBOSExecutor.close() returned at its first exception, so a Conductor client or lifecycle listener that threw on stop left the queue service, scheduler, system database pool and thread pools running. A failed launch now cleans up through close() and then clears the executor, so a retry could run next to them. close() now runs every step, then throws the first failure with the rest attached. DBOS.shutdown() clears the executor even when close() throws, and a failed launch no longer tries to attach an exception to itself when close() throws the same one.
kraftp
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When
DBOS.launch()throws, the instance is now left as iflaunch()had never been called: whatever the failed launch started is shut down, and a laterlaunch()tries again from the start.Why
launch()stored its executor in the instance before running the migrations and starting it. When either threw, the instance kept that unstarted executor:launch()returned at once, with nothing running;NullPointerException(this.systemDatabase is null) on what it never started.Every deployment that runs with
withMigrate(false)against a system database below the required version takes this path. So does any other failure during launch: an unreachable database, a failed migration, a lifecycle listener that throws, a Conductor provider that fails to load. A 1.2.0 to 1.3 upgrade test found it.How
DBOS.launch()wraps the migrations or version check andDBOSExecutor.start(). On a failure it closes the executor, clears it and rethrows. A failure to close is attached as suppressed.DBOSExecutor.close()no longer throws an NPE whenstart()failed before creating its executor service and timeout scheduler.launch()'s javadoc says what a failed launch leaves behind.Tests
New
FailedLaunchTest; both tests fail without the fix:🤖 Generated with Claude Code