Skip to content

feat(macos): load COPY FROM STDIN rows through copy-in in SQL import and the editor - #4255

Merged
datlechin merged 2 commits into
mainfrom
feat/d-copyin
Nov 13, 2026
Merged

datlechin merged 2 commits into
mainfrom
feat/d-copyin

Conversation

@datlechin

@datlechin datlechin commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

  • D-7: COPY ... FROM STDIN loads its rows through libpq's copy-in protocol, in SQL import and in the query editor, as psql sends a script's rows: CopyData pieces, never the \. line (PostgreSQL 18 reads it as data in CSV), then CopyDone. Text and CSV load; binary refuses with CopyFail, no rows sent.
  • PluginKit (additive, kit stays 44): PluginCopyData (pull a piece at a time), PluginCapabilities.copyFromStandardInput, PluginDatabaseDriver.copyFromStandardInput(statement:rows:context:), PluginImportElement, PluginImportSource.elements() (defaults to statements()), PluginImportDataSink.copyFromStandardInput(statement:line:rows:).
  • PostgreSQL plugin: one dispatch holds the connection from the COPY to its last result, so nothing else can end the copy. Stop or a failing source ends it with CopyFail and reports that, not the server's echo. Under Read-Only Safe Mode it runs in the read-only transaction, which the server refuses (25006) before any row. Declared on PostgreSQL, CockroachDB and KingbaseES; not on Redshift (no COPY FROM STDIN) or PGlite (pglite-socket 0.2.19 drops the connection on one, psql's too).
  • SQL import: the parser yields the COPY with a reader over the file, a piece (at most 1 MiB) at a time, inside the import's transaction; rows left unread are passed over. The progress line counts lines sent. statements() still refuses a COPY for plugins built before this. A driver without copy-in refuses at the COPY's line.
  • Editor: a statement carries its COPY rows (ExecutableStatement.copyData) and runs on the statement path, never the single or batch path. A driver without copy-in refuses before anything is sent. MCP, the assistant and AppleScript refuse a COPY FROM STDIN with a message naming the editor and import.
  • Rust core (Linux, Windows): no change. Its SQL import already loads pg_dump's text COPY rows as INSERTs (ADR 0011) and refuses CSV or options; its editor refuses COPY FROM STDIN. Copy-in there would go through tokio-postgres 0.7.18 copy_in, which leaves the connection out of step after a COPY the server refuses at Execute (measured against PostgreSQL 17 in docker), so a missing table, a permission error or a Stop during a lock wait would end the session.
  • New probe: apple/scripts/probes/check-postgresql-copy-in.sh (libpq 17.4 from apple/Libs against a live server).
  • Tests changed on purpose: SQLFileParserTests.postgresCopyFromStandardInputIsRefused is now postgresStatementStreamRefusesCopyFromStandardInputAndCountReadsIt: the statement stream still refuses at the COPY, and the count now counts it.
  • Docs: docs(features): load COPY FROM STDIN rows in SQL import and the editor docs#230

Test plan

  • New: LibPQCopyInTests (scripted server: bytes in order, no \., COPY 4, CopyFail on a source error and on Stop, binary refused before a piece is read, a server refusal, Read-Only wrap), SQLImportCopyRowsTests, ImportDataSinkAdapterCopyTests, SQLFileParserCopyRowsTests (LF, CRLF, no \., chunk and piece boundaries, non-ASCII, unread rows, counts), CopyDataRouteTests, CopyFromStandardInputRunTests, DatabaseAccessBridgeCopyTests, SQLPsqlScriptTests additions.
  • Measured with check-postgresql-copy-in.sh: libpq 17.4 against PostgreSQL 17.11 and 18.6, 21 checks each. PGlite 0.5.16 through pglite-socket 0.2.19 with psql 17.11.

Needs the maintainer

  • Rust core copy-in: built and tested (docker, pgbouncer included), then held back for the cost above. It needs tokio-postgres to send nothing on a copy_in the server refused before CopyInResponse (upstream fix or a [patch.crates-io]), or the cost accepted. The patch is kept for a follow-up.
  • Stop during a long COPY asks its question once the COPY ends, as for any statement: ending it early would lose the block, so Continue could not resume it.
  • The import's progress bar counts statements, and a COPY is one, so it holds still through a large block; the line under it counts the lines sent. A byte-based bar would change every SQL import.
  • CockroachDB and KingbaseES declare copy-in through the plugin's shared libpq code; neither was measured here (no server on this Mac).

@datlechin
datlechin enabled auto-merge (squash) November 12, 2026 22:39
@datlechin
datlechin merged commit 9d15935 into main Nov 13, 2026
11 of 13 checks passed
@datlechin
datlechin deleted the feat/d-copyin branch November 14, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant