Run onexecute regardless of max_pipeline so begin() works at 0 - #1211
reinierlakhan wants to merge 1 commit into
Conversation
execute() returned a single && chain that did two unrelated jobs:
deciding whether the pool may pipeline another query onto this
connection, and running the query's onexecute hook. begin() relies on
that hook to capture the connection and move it to the reserved queue.
With max_pipeline: 0 the `sent.length < max_pipeline` term is always
false, so the chain short-circuited before onexecute ran. The connection
was never reserved and the BEGIN's CommandComplete guard rejected every
transaction with UNSAFE_TRANSACTION.
Run the hook whenever the query was actually written, and keep the
return value's meaning ("may the pool pipeline more onto this
connection") unchanged. For max_pipeline > 0 behaviour is identical:
onexecute returns a truthy value, so the old chain was truthy exactly
when the new one is. For max_pipeline: 0 execute() now returns false
after reserving, the pool moves the connection to full, and on the
BEGIN's ReadyForQuery connection.reserved() drains the transaction's own
queue or moves it back to reserved, so nothing is pipelined.
|
@reinierlakhan I had written a similar fix, just PRed it on #1218. I believe there's a regression in your branch that causes another query to end up in the connection that should be reserved for the transaction. I wrote a test reproducing it here |
|
With the default The PR description says Repro: plain PostgreSQL 16, import postgres from 'postgres'
for (const max_pipeline of [undefined, 1, 0]) {
let beginWritten
const begun = new Promise(r => (beginWritten = r))
const sql = postgres(process.env.DATABASE_URL, {
max: 1,
...(max_pipeline === undefined ? {} : { max_pipeline }),
debug: (_, q) => /^begin/i.test(q.trim()) && beginWritten()
})
await sql`select 1`
const tx = sql.begin(async t => (await t`select now()::text as ts, pg_sleep(0.3)`)[0].ts)
await begun
const outside = sql`select now()::text as ts`
const [txTs, [row]] = await Promise.all([tx, outside])
console.log(`max_pipeline=${max_pipeline ?? 'default'}: outside query ran inside the transaction = ${row.ts === txTs}`)
await sql.end()
}
The #1211 result reproduced 3 out of 3 times. #1218 keeps |
Fixes #1210
Cause
execute(q)insrc/connection.jsreturns one&&chain that does two unrelated jobs: it decides whether the pool may pipeline another query onto this connection, and it runs the query'sonexecutehook as its last term:begin()sendsBEGINwith{ onexecute }and relies on that hook to capture the connection,move(c, reserved)and setc.reserved. Withmax_pipeline: 0thesent.length < max_pipelineterm is always false, so the chain short-circuits before the hook. The connection is never reserved, and theUNSAFE_TRANSACTIONguard inCommandCompleterejects every transaction.Change
Run the hook whenever the query was actually written, independent of pipeline capacity, and keep the return value's meaning ("may the pool pipeline more onto this connection") unchanged:
max_pipeline > 0: behaviour is identical.onexecuteinbegin()(the only caller) returns a truthy value (the assignedc.reservedfunction), so the old chain was truthy exactly when the new one is, and the hook ran in exactly the cases it runs now (previously it also only ran whenwrite()returned truthy, whichwritten &&preserves).max_pipeline: 0:execute()reserves the connection and returnsfalse, sogo()moves it tofull.c.reservedis set, so the BEGIN passes the guard. On the BEGIN's ReadyForQuery (statusT),connection.reserved()runs, finds the transaction's queue empty and moves the connection back toreserved. The transaction callback only starts after that (it is awaited behind the BEGIN), and each statement it issues goes throughbegin()'shandler:execute()returnsfalse->move(c, full); anything issued while infullis pushed onto the transaction's ownqueriesand drained one at a time byc.reserved()on each ReadyForQuery. Ordering is preserved and nothing is pipelined onto the socket, which is the point of the setting.Tests
Three tests added to
tests/index.js, each on a client with{ max: 4, max_pipeline: 0 }:sql.begin(sql => sql\select 1 as x`)returns1`set_config(..., true)thencurrent_setting, so it also proves they ran on the same transaction)queriesqueue pathRun in isolation against a plain PostgreSQL 16 through the repo's
tests/test.jsharness: all three pass with this change; without it the first fails withUNSAFE_TRANSACTION.eslint src testswith the repo config is clean.The full suite did not run here:
tests/bootstrap.jsneedspsql/createdb/dropdbon PATH and a server it may reconfigure (ssl=on,wal_level=logical, prepared transactions), which this environment does not have. CI should cover it.I left
cjs/,deno/andcf/untouched, following the pattern of separatebuildcommits in this repo; happy to regenerate them in this PR if preferred.