Skip to content

Fix binary NUMERIC-to-DOUBLE rounding - #610

Merged
staticlibs merged 2 commits into
duckdb:mainfrom
ItsM0rty:fix/numeric-double-rounding
Oct 13, 2026
Merged

staticlibs merged 2 commits into
duckdb:mainfrom
ItsM0rty:fix/numeric-double-rounding

Conversation

@ItsM0rty

@ItsM0rty ItsM0rty commented Oct 13, 2026 •

Copy link
Copy Markdown

Fixes #604.

Unconstrained NUMERIC values and NUMERIC types outside DuckDB's DECIMAL limits are read as DOUBLE. The binary decoder currently accumulates their base-10000 digits with floating-point arithmetic, rounding at several intermediate steps. For example, PostgreSQL's 8629.433::numeric can be read as 8629.432999999999. Applying the display scale can also overflow while decoding finite values such as 1.5::numeric(1000,500).

This builds a decimal coefficient from the stored digits with exponent 4 * (weight + 1 - ndigits), then calls DuckDB's floating-point parser once. There is no need to expand implicit zero groups or apply the display scale with floating-point arithmetic. The handling of zero, NaN, infinity, and constrained DECIMAL values stays the same.

The new sqllogictest compares decoded values against PostgreSQL's float8 cast in COPY_BINARY, PLAIN_QUERY_BINARY, and PLAIN_QUERY_TEXT modes. It covers the reported values, halfway rounding, finite limits, subnormals, high display scales, NULL, special values, and NUMERIC arrays. Results are materialized before comparison so predicate pushdown cannot hide a decoder error.

Validation:

  • The original extension fails the regression with 15 mismatches in the first 24-value dataset. The fix passes all 23 regression assertions.
  • numerics_nan_inf.test, attach_types_decimal.test, and attach_types_numerics.test pass, with 295 assertions in total.
  • Built DuckDB, the connector, CLI, and SQL test runner with Clang 18.1.3 on Ubuntu 24.04. Tests used PostgreSQL 16.15 and libpq 18.6. The local build used O1 without debug information to fit WSL memory; C++ API tests and unrelated extensions were disabled.
  • clang-format 11.0.1 and whitespace checks pass.

CI also caught an existing giant-NUMERIC array expectation that recorded the old rounding error. The giant-NUMERIC and isolation-level expectations now use the corrected values (1e+32 and -1.2345678912345679e+26), checked against PostgreSQL's float8 and float8[] casts. Those two test files and the rounding regression pass locally with 50 assertions.

I haven't run the full connector suite, cross-platform tests, malformed-wire-input tests, or throughput benchmarks.

Preserve PostgreSQL base-10000 digits in a scientific-notation string and convert once with DuckDB floating-point parsing. Avoid repeated rounding and intermediate overflow from large display scales.

Add exact comparisons against PostgreSQL float8 across binary COPY, binary query, and text query modes, including halfway values, subnormals, high-scale decimals, and arrays. Fixes duckdb#604.

@staticlibs staticlibs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks good to me.

@staticlibs
staticlibs merged commit ae8ae90 into duckdb:main Oct 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NUMERIC read as DOUBLE doesn't match Postgres's float8 cast

2 participants