Conversation
sql.begin() could send BEGIN to the server without reserving the connection. Depending on pool size, this errored with UNSAFE_TRANSACTION or TypeError. The transaction stayed open on the server either way, and the connection went back to the pool, so unrelated queries then ran inside it. With max_pipeline: 0 this error happened consistently. With the default max_pipeline of 100 the same thing happened only when BEGIN was the query that filled a connection's pipeline, which is why the failure was intermittent. The issue is in execute(): after writing the query it returns an && chain that states whether the connection can accept more queries, and one of its terms is sent.length < max_pipeline. The onexecute hook, which is how sql.begin() reserves the connection, was the last term of that chain. Whenever the capacity term was false the chain short-circuited and the hook never ran, even though BEGIN had already been written. With max_pipeline: 0 that term is false for every query, so the hook never ran at all. Move the hook ahead of the capacity terms so it runs whenever the query was written. Reserving the connection is a consequence of having sent BEGIN, unrelated to the query limit in the connection. Add tests for BEGIN at the pipeline boundary and with max_pipeline: 0. Fixes porsager#1189
execute() must keep returning a falsy value for BEGIN so the pool puts the connection in the full queue rather than busy. Otherwise a query issued while BEGIN is in flight is pipelined behind it and runs inside the transaction.
|
A production data point for this fix, and for #970 / #1210.
We run that patch in production. Its semantics match this PR: call the hook whenever the bytes were written, and keep a falsy return while it's present. In our local tests it passes concurrent transactions, savepoints and rollback at |
|
+1 for this approach — we hit the same bug in our app and landed on Setup: postgres 3.4.8 (code unchanged in 3.4.9), Node 24.13, Drizzle, Why we need Then Our patch keeps the falsy return for queries carrying build(q)
const pipelinable = write(toBuffer(q))
&& !q.describeFirst
&& !q.cursorFn
&& sent.length < max_pipeline
return q.options.onexecute
? q.options.onexecute(connection)
: pipelinableOne small difference from this PR: we run the hook even when Results ( Happy to turn this into a test for |
sql.begin() could send BEGIN to the server without reserving the
connection.
Depending on pool size, this errored with UNSAFE_TRANSACTION or
TypeError. The transaction stayed open on the server either way, and
the connection went back to the pool, so unrelated queries then ran
inside it.
With max_pipeline: 0 this error happened consistently. With the default
max_pipeline of 100 the same thing happened only when BEGIN was the
query that filled a connection's pipeline, which is why the failure was
intermittent.
The issue is in execute(): after writing the query it returns an &&
chain that states whether the connection can accept more queries, and
one of its terms is sent.length < max_pipeline. The onexecute hook,
which is how sql.begin() reserves the connection, was the last term of
that chain. Whenever the capacity term was false the chain
short-circuited and the hook never ran, even though BEGIN had already
been written. With max_pipeline: 0 that term is false for every query,
so the hook never ran at all.
Move the hook ahead of the capacity terms so it runs whenever the query
was written. Reserving the connection is a consequence of having sent
BEGIN, unrelated to the query limit in the connection.
Add tests for BEGIN at the pipeline boundary and with max_pipeline: 0.
Fixes #1189 #1210