Skip to content

fix(core): end a DuckDB name where DuckDB's pass turns a non-ASCII space into a space - #4239

Merged
datlechin merged 2 commits into
mainfrom
fix/audit-w6-core-lexer
Nov 12, 2026
Merged

datlechin merged 2 commits into
mainfrom
fix/audit-w6-core-lexer

Conversation

@datlechin

Copy link
Copy Markdown
Member

Summary

  • DuckDB 2.0.0 turns U+00A0, U+2000 to U+200B, U+202F, U+205F, U+2060, U+3000 and U+FEFF into spaces before it lexes, except inside what its own pass reads as a quote. The core lexer read them as part of a name, so SELECT varchar + U+200B + E'\''; DROP TABLE t; --' was one statement to Safe Mode and two to DuckDB.
  • The core lexer now runs that pass, mirrored from DuckDB's Parser::StripUnicodeSpaces, from each statement's first code token, where the text the driver sends DuckDB starts.
  • Safe Mode on Linux and Windows, MCP's one-statement check, the history's secret check and the catalog refresh also read DuckDB text without the pass and keep the stricter answer.
  • The DuckDB driver splits such text as DuckDB does, so an exported query's read-only transaction refuses a hidden COMMIT.
  • The ATTACH guard now evaluates the path as DuckDB reads it in the ATTACH, and refuses when its own query would read it otherwise. Before, it could run a hidden DROP, or check a+U+00A0+b.duckdb while DuckDB opened a b.duckdb that another window had open.
  • PostgreSQL already ends a name where the server does in the core (x$e'\' is a name and a plain string); a test pins it. Oracle and Dameng have no grammar in the core.

Test plan

  • New, failing on main: script_corpus DuckDB cases, statement_access (a text where DuckDB keeps one U+200B and blanks another was Read), the DuckDB driver's split, controls_transaction and main_verb, the export integration test (the hidden INSERT ran), two ATTACH integration tests (the table was dropped; the held file was attached), the DuckDB history test, MCP single_batch. New, passing on main: the PostgreSQL case and server_reading.
  • Measured on the DuckDB 2.0.0 library (source d54cbbe3df, as the bundled crate) and PostgreSQL 17.11.
  • The mirror against DuckDB 2.0.0's own compiled pass on 191,684 random texts: no difference.
  • verify.sh core for tablepro-core, tablepro-driver-duckdb, tablepro-mcp and tablepro-ffi; clippy and rustfmt; Windows build and test. Two DuckDB driver tests fail on macOS before and after (/proc/self/clear_refs, /private/var paths).

@datlechin
datlechin merged commit efbf898 into main Nov 12, 2026
1 check passed
@datlechin
datlechin deleted the fix/audit-w6-core-lexer branch November 12, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant