Repository navigation
feat(macos): load COPY FROM STDIN rows through copy-in in SQL import and the editor - #4255
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COPY ... FROM STDINloads its rows through libpq's copy-in protocol, in SQL import and in the query editor, as psql sends a script's rows:CopyDatapieces, never the\.line (PostgreSQL 18 reads it as data in CSV), thenCopyDone. Text and CSV load; binary refuses withCopyFail, no rows sent.PluginCopyData(pull a piece at a time),PluginCapabilities.copyFromStandardInput,PluginDatabaseDriver.copyFromStandardInput(statement:rows:context:),PluginImportElement,PluginImportSource.elements()(defaults tostatements()),PluginImportDataSink.copyFromStandardInput(statement:line:rows:).COPYto its last result, so nothing else can end the copy. Stop or a failing source ends it withCopyFailand reports that, not the server's echo. Under Read-Only Safe Mode it runs in the read-only transaction, which the server refuses (25006) before any row. Declared on PostgreSQL, CockroachDB and KingbaseES; not on Redshift (noCOPY FROM STDIN) or PGlite (pglite-socket 0.2.19 drops the connection on one, psql's too).COPYwith a reader over the file, a piece (at most 1 MiB) at a time, inside the import's transaction; rows left unread are passed over. The progress line counts lines sent.statements()still refuses aCOPYfor plugins built before this. A driver without copy-in refuses at theCOPY's line.COPYrows (ExecutableStatement.copyData) and runs on the statement path, never the single or batch path. A driver without copy-in refuses before anything is sent. MCP, the assistant and AppleScript refuse aCOPY FROM STDINwith a message naming the editor and import.COPYrows as INSERTs (ADR 0011) and refuses CSV or options; its editor refusesCOPY FROM STDIN. Copy-in there would go through tokio-postgres 0.7.18copy_in, which leaves the connection out of step after aCOPYthe server refuses at Execute (measured against PostgreSQL 17 in docker), so a missing table, a permission error or a Stop during a lock wait would end the session.apple/scripts/probes/check-postgresql-copy-in.sh(libpq 17.4 fromapple/Libsagainst a live server).SQLFileParserTests.postgresCopyFromStandardInputIsRefusedis nowpostgresStatementStreamRefusesCopyFromStandardInputAndCountReadsIt: the statement stream still refuses at theCOPY, and the count now counts it.Test plan
LibPQCopyInTests(scripted server: bytes in order, no\.,COPY 4, CopyFail on a source error and on Stop, binary refused before a piece is read, a server refusal, Read-Only wrap),SQLImportCopyRowsTests,ImportDataSinkAdapterCopyTests,SQLFileParserCopyRowsTests(LF, CRLF, no\., chunk and piece boundaries, non-ASCII, unread rows, counts),CopyDataRouteTests,CopyFromStandardInputRunTests,DatabaseAccessBridgeCopyTests,SQLPsqlScriptTestsadditions.check-postgresql-copy-in.sh: libpq 17.4 against PostgreSQL 17.11 and 18.6, 21 checks each. PGlite 0.5.16 through pglite-socket 0.2.19 with psql 17.11.Needs the maintainer
copy_inthe server refused beforeCopyInResponse(upstream fix or a[patch.crates-io]), or the cost accepted. The patch is kept for a follow-up.COPYasks its question once theCOPYends, as for any statement: ending it early would lose the block, so Continue could not resume it.COPYis one, so it holds still through a large block; the line under it counts the lines sent. A byte-based bar would change every SQL import.