Skip to content

feat(core): load imported COPY rows through copy-in on a fixed tokio-postgres - #4258

Merged
datlechin merged 3 commits into
mainfrom
feat/d-core-copyin
Nov 13, 2026
Merged

datlechin merged 3 commits into
mainfrom
feat/d-core-copyin

Conversation

@datlechin

@datlechin datlechin commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Import SQL File on PostgreSQL sends the rows of a COPY ... FROM stdin block through the new EditorSession::copy_in, piece by piece as the file is read, in the import's transaction or one of the driver's own. Text and CSV load with any option. Binary is refused and its rows skipped. ADR 0024; ADR 0011's INSERT path is superseded.
  • tokio-postgres 0.7.18 sends a Sync with the COPY's Execute and another with its CopyFail, so a COPY the server refuses before copy-in mode (missing table, permission, Stop during a lock wait) left the connection out of step. The workspace takes tokio-postgres, postgres-types and postgres-protocol from TableProApp/rust-postgres (the 0.7.18 tag plus fix(tokio): flush instead of sync when copy_in starts rust-postgres/rust-postgres#1402, Flush in place of that first Sync) through [patch.crates-io]. deny.toml allows that git source. The license files do not change.
  • The RPM spec appends the source replacement cargo vendor writes for those git crates, which %cargo_prep leaves out; the Debian and Flatpak builds already copy it.
  • Such a COPY now fails like any statement and the session goes on. Under Continue the import goes on.
  • A read-only session runs the copy in START TRANSACTION READ ONLY, and the server refuses it with 25006.
  • A COPY refused after its end was already queued no longer drops the statement after its block.
  • The script reader starts the rows on the line after the COPY's ;, runs what follows the COPY on that line once the rows end, as psql does, and waits for the rest of a long COPY line before it reads rows.
  • A failed import statement shows the server's context, which names the row of a failed COPY.
  • The statements read whole before a byte that is not UTF-8, a cut-off last character or a read error now run before the import stops.
  • Linux error texts for binary COPY rows and for a session that cannot take COPY rows; README, roadmap and driver docs updated.
  • Tests that pinned the old INSERT path, changed here: pg_dump_copy_rows_become_inserts_and_restrict_lines_are_passed_over (now ..._follow_their_copy_...), a_copy_with_options_is_refused_and_its_rows_skipped (now a_csv_copy_with_options_passes_its_rows_as_written), and the removed a_text_format_line_becomes_literals, octal_and_hex_escapes_read_as_bytes, an_empty_field_is_an_empty_string.
  • Docs: the docs site has no page for Linux SQL import; linux/README.md is its home.

Test plan

  • Docker, tablepro-driver-postgres: copy_in::* (11), pgbouncer::transaction_pool_copy_* (2), dump::suite::native_dump_loads_through_the_import, csv_import::*. The five refused or stopped COPY tests fail on unpatched 0.7.18 and pass here.
  • sql_import_copy_in::a_copy_refused_after_its_end_was_queued_keeps_the_statement_after_it fails on the first version of the import loop; a_copy_on_a_long_line_waits_for_the_rest_of_its_line and the_statements_read_whole_before_the_file_stops_being_readable_run fail without their fixes.
  • cargo test for tablepro-core, tablepro-test-support, tablepro-driver-postgres and tablepro-session; clippy with -D warnings; cargo deny check bans licenses sources; update-third-party-licenses.sh writes no change.
  • Upstream: copy_in_statement_error and copy_in_not_a_copy fail on 0.7.18; all three new tests and the crate suite pass with the fix.

@datlechin
datlechin enabled auto-merge (squash) November 13, 2026 01:33
@datlechin
datlechin merged commit 36ef919 into main Nov 13, 2026
46 of 47 checks passed
@datlechin
datlechin deleted the feat/d-core-copyin branch November 14, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant