Skip to content

fix(proto): store far-future DateTime64 without int64 overflow - #1184

Open
dogacid wants to merge 1 commit into
ClickHouse:mainfrom
global-roam:gr/datetime64-full-range
Open

fix(proto): store far-future DateTime64 without int64 overflow#1184
dogacid wants to merge 1 commit into
ClickHouse:mainfrom
global-roam:gr/datetime64-full-range

Conversation

@dogacid

@dogacid dogacid commented Aug 6, 2026

Copy link
Copy Markdown

Summary

proto.ToDateTime64 and DateTime64.Time convert a timestamp through an int64 nanosecond
count (t.UnixNano() on write, int64(d) * p.Scale() on read). That intermediate overflows int64
just past 2262-04-11 23:47:16 UTC, so any later timestamp is silently corrupted — on the way in
and out — even for precisions whose stored tick would fit int64 across a far wider range.

ClickHouse 26.7 extended the DateTime64 range (millisecond precision now stores up to
9999-12-31), so this is reachable in ordinary use. Concretely, writing an open-ended sentinel such
as 9999-12-31 23:59:59.999 into a DateTime64(3) column currently round-trips as 1900-… garbage.

Root cause

A DateTime64(P) value is stored as an int64 count of ticks of 10^-P seconds. The conversions
first scale the whole timestamp to nanoseconds:

// write
return DateTime64(t.UnixNano() / p.Scale())
// read
nsec := int64(d) * p.Scale()

t.UnixNano() (and the * p.Scale() on read) overflow int64 at ~2262 — even though the final tick
count at, say, millisecond precision would not overflow until roughly the year 292 million.

Fix

Compute at the column's own tick scale instead of via nanoseconds, so no oversized intermediate is
ever formed:

// write
secScale := int64(1e9) / p.Scale() // ticks per second (10^P)
return DateTime64(t.Unix()*secScale + int64(t.Nanosecond())/p.Scale())
// read
secScale := int64(1e9) / p.Scale()
return time.Unix(int64(d)/secScale, (int64(d)%secScale)*p.Scale())
  • Identical result for all currently-working values — it is the same arithmetic, reordered so the
    intermediate never exceeds the tick magnitude.
  • Each precision now reaches its true range — the same range ClickHouse's int64-tick
    DateTime64(P) supports. Coarser precisions reach much further; nanosecond precision (9) still
    caps near 2262
    , which is the genuine int64-nanosecond limit and matches ClickHouse's own
    DateTime64(9).
  • The t.IsZero() short-circuit is preserved.

Tests

Adds TestDateTime64_FarFuture: round-trips the 2262 boundary, 2999, and the 9999-12-31 ceiling
across precisions 0/1/3/6/7, plus a millisecond-fraction case. These fail on main today
(e.g. 9999-12-31 → 1816-03-30) and pass with the fix. Existing tests are unchanged.

Notes

No measurable performance impact (a couple of extra integer operations; no allocation).

ToDateTime64 and DateTime64.Time convert through t.UnixNano() / an int64
nanosecond count, which overflows just past 2262-04-11 and corrupts any later
timestamp on both the write and read paths (e.g. ClickHouse 26.7's 9999-12-31
ceiling for DateTime64(3) was written back as ~1900).

Compute the tick count at the column's own scale instead, so each precision
reaches exactly ClickHouse's DateTime64(precision) range. Nanosecond precision
still caps near 2262 where an int64 count of nanoseconds runs out, matching
ClickHouse itself; the result is identical for all in-range values.

Add far-future round-trip tests covering the 2262 boundary and the 9999 ceiling
across precisions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant