Skip to content

fix(duckdb): keep DuckDB's expression recursion within the stacks it runs on - #4293

Merged
datlechin merged 5 commits into
mainfrom
fix/duckdb-deep-check-crash
Nov 15, 2026
Merged

datlechin merged 5 commits into
mainfrom
fix/duckdb-deep-check-crash

Conversation

@datlechin

@datlechin datlechin commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Every DuckDB database the apps open on a user's file sets max_expression_depth to 250, so DuckDB refuses a deeper statement with its own Parser or Binder error instead of overflowing a stack. A CHECK about 575 deep crashed the app on macOS when any table's keys were read: duckdb_constraints() binds every CHECK on DuckDB's own worker threads, which get 512 KiB there.
  • Core: every DuckDB call, and the close that checkpoints the file, runs on an 8 MiB thread (on_duckdb_stack), the stack DuckDB's CLI has, so the open's unguarded catalog read no longer overflows the runtime's 2 MiB blocking thread. The app's own connections pin the limit per session, so an editor tab's SET GLOBAL does not reach them.
  • macOS DuckDB plugin and the iOS driver: the same limit, and the DuckDB actor runs on an 8 MiB thread through a custom executor (DuckDBStackExecutor, shared by both apps), so an open, an ATTACH or a close of a deep file no longer overflows a 512 KiB cooperative or dispatch thread. The macOS isolated import opens and closes its file on that stack.
  • core.yml runs the new DuckDB depth tests on Windows x64, Windows Arm64 and macOS, where DuckDB's threads get other stacks than on Linux.
  • Core ADR 0031 records the measured stack per level, the limit, the rejected design (owning DuckDB's threads through external_threads) and what stays DuckDB's: a file built past DuckDB's own limit can still overflow DuckDB's threads where it prints or saves expressions.
  • SQLite: no change. The bundled 3.51.3 grows its parser stack on the heap and limits expression depth to 1,000; at that depth and with a 3,000-view chain it refuses on a 2 MiB stack.

Test plan

  • The deep_expressions test target, which core.yml also runs on Windows and macOS:
    • a_file_as_deep_as_duckdb_allows_is_read_or_refused: a CHECK, DEFAULT, generated column, view and macro 992 deep (DuckDB's own deepest); reads are refused at the limit. Fails on main.
    • a_file_deeper_than_a_runtime_thread_holds_opens: a CHECK 3,000 deep opens; on main the open dies with SIGILL.
    • the_limit_holds_on_every_connection_after_a_tab_raises_it: an editor tab and the workspace refuse 250 levels, the workspace also after the tab's SET GLOBAL; fails without the session pin.
  • iOS DuckDBDeepExpressionTests: a CHECK 900 deep opens and reading its keys fails at the limit; an ATTACH of it reads its catalog.
  • verify.sh core tablepro-driver-duckdb (the 2 macOS-only procfs and /private path cases fail on main too), plugins, ios, lint.

@datlechin
datlechin merged commit 2aca62e into main Nov 15, 2026
34 checks passed
@datlechin
datlechin deleted the fix/duckdb-deep-check-crash branch November 17, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant