(janitor/flaky-test) fix unhandled pg Pool error in admin-permissions-migration test - #5769
Merged
Merged
Conversation
…-migration test The migration test creates its own `pg.Pool` instances and force-drops its scratch database in the `finally` block. `Pool#end()` resolves once the client sockets are told to close, but the server's `DROP DATABASE ... WITH (FORCE)` termination can still land on a socket that has not finished closing. node-postgres re-emits that as an 'error' event on the pool, and with no listener attached Node treats it as an unhandled error, failing whichever test happens to be running when it surfaces. The app's own pools in drizzle.ts already attach an 'error' listener for exactly this reason (see the comment there). Do the same for the pools this test creates.
Contributor
Author
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (1 files)
Reviewed by grok-4.6 · Input: 56K · Output: 4.8K · Cached: 97.5K Review guidance: REVIEW.md from base branch |
pandemicsyn
approved these changes
Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flaky test
apps/web/src/lib/admin/admin-permissions-migration.test.ts>admin permissions migration › backfills only admins that matched the previous grant-authority predicateEvidence
9293124762296beb3f614efa2cb365260a46644f) with:test-job failure across the last 120 failingCIruns on this repo (gh run list --workflow=CI,08-29through08-31). All othertest/typecheck/lintfailures were either thedependency-cycle-check(madge) step misreported under thetestjob name, ortypecheck/lintfailures — none were a second occurrence of this or any other named-test failure, i.e. this test fails intermittently and is the only reproducible named-test flake in that window (frequency: 1 observed failure across ~890test-job runs of this suite in the sampled window).FATAL: terminating connection due to administrator command(57P01) surfacing as a Jest "Unhandled error" rather than a normal assertion failure — i.e. the test's own logic never disagreed with expectations.Root cause
This test:
testPool = new Pool(...)connected to it.finallyblock, callsawait testPool.end()and then immediatelyDROP DATABASE ... WITH (FORCE)against that same database.Pool#end()resolves once the underlying client sockets are told to close, but does not guarantee the server has finished processing that close.DROP DATABASE ... WITH (FORCE)force-terminates any backend still associated with the target database at the moment it runs. When the socket teardown and the forced termination race — which depends on runner/network timing and is therefore nondeterministic — the termination FATAL can still arrive on a socket the pool considers already closing.pg-poolre-emits that as an'error'event on thePool. NeitheradminPoolnortestPoolin this test had an'error'listener attached, so Node treats the event as an unhandled error, which Jest attributes to whichever test happens to be executing when it lands — producing an intermittent, timing-dependent failure unrelated to the test's assertions.This is exactly the scenario the app's own primary/replica pools already guard against in
apps/web/src/lib/drizzle.ts:This test's ad-hoc
pg.Poolinstances never got the same treatment.Fix
Attach an
'error'listener (log-and-continue, matching the production pattern) to bothadminPoolandtestPoolin this test. This does not change what the test asserts, add retries/timeouts/sleeps, or touch production code — it only ensures that a raced, already-expected idle-client error during teardown surfaces as a handled event instead of crashing the test run.Why this is deterministic
The fix does not depend on timing: an
'error'listener on aPoolis always safe to attach and always prevents the "unhandled error" crash regardless of whether the race actually happens on a given run. The test's real assertions (migration backfill behavior) are unchanged and continue to run to completion before the pools are torn down.Validation
pnpm exec oxlint --config .oxlintrc.json apps/web/src/lib/admin/admin-permissions-migration.test.ts→ 0 warnings/errors.pnpm exec oxfmt --list-different apps/web/src/lib/admin/admin-permissions-migration.test.ts→ no diff.apps/web:pnpm exec tsgo --noEmit→ clean.