Defer DB connection until after jixia analysis completes#17
Merged
Conversation
TCP keepalives (previous fix) did not prevent the connection drop -- same failure recurred at the same spot. Keepalives are invisible to whatever is actually closing the connection (likely an application-level idle policy on Heroku's side, not a network path issue), so they can't help. Split jixia_db.load_data into run_analysis() (pure CPU work, no DB connection) and load_data() (DB writes only). database/__init__.py now runs the 30-90+ minute analysis phase first, then opens a fresh connection right before writing -- so no connection is ever left idle for more than the time between individual queries.
There was a problem hiding this comment.
Pull request overview
This PR restructures the jixia ingestion pipeline so the long, CPU-only jixia analysis phase runs before any PostgreSQL connection is opened, preventing idle-connection drops (observed on the first INSERT INTO module after analysis) during CI/Heroku runs.
Changes:
- Split
database.jixia_db.load_dataintorun_analysis(...)(CPU-only) andload_data(..., analysis, conn)(DB writes only). - Updated CLI dispatch to run analysis first, then open a DB connection only for the write phase.
- Factored DB connection creation into a shared
_connect()helper and moved connection opening into each command branch.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
database/jixia_db.py |
Introduces run_analysis() and refactors load_data() to consume precomputed analysis results rather than running jixia while holding a DB connection. |
database/__init__.py |
Defers DB connection creation until just before DB-write phases; adds _connect() and updates command dispatch accordingly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The TCP keepalive fix (#16) did not help — the run failed identically, at the same spot (
Units/WithDim/Mass, the firstINSERT INTO moduleafter analysis), with the same error:This means the connection isn't being dropped by a network-level idle timeout that keepalive packets would prevent — something (most plausibly Heroku's own connection handling) is closing it based on application-level inactivity, which TCP keepalives are invisible to.
What
Stop trying to keep a connection alive through the long idle period at all. Split
jixia_db.load_datainto:run_analysis(project, prefixes)— pure CPU work (batch_run_jixia), no DB connection, can take 30-90+ min.load_data(project, analysis, conn)— DB writes only, using the precomputed analysis.database/__init__.pynow runs the full analysis first, and only opens the (still keepalive-enabled, cheap to keep) DB connection right before the write phase begins. No connection is ever left idle for more than the gap between individual queries.Also factored connection setup into a
_connect()helper and moved connection-opening into each command branch (previously one connection wrapped all four commands, opened before argument dispatch even mattered) —schema/informal/vector-dbare unaffected since they were already fast.