Skip to content

Fix Timestamp.from_datetime() precision for far-future datetimes#710

Merged
methane merged 1 commit into
msgpack:mainfrom
gaoflow:fix-from-datetime-far-future-precision
Jul 5, 2026
Merged

Fix Timestamp.from_datetime() precision for far-future datetimes#710
methane merged 1 commit into
msgpack:mainfrom
gaoflow:fix-from-datetime-far-future-precision

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Timestamp.from_datetime() computes the whole-second part from the float datetime.timestamp(). A float64 cannot hold microsecond precision for datetimes far from the epoch, so timestamp() rounds the seconds up while the exact microsecond is still used for the nanoseconds. The result is a Timestamp one second in the future, and near datetime.max it raises OverflowError:

>>> import datetime as dt
>>> from msgpack.ext import Timestamp
>>> d = dt.datetime(3000, 1, 1, 0, 0, 0, 999999, tzinfo=dt.timezone.utc)
>>> Timestamp.from_datetime(d).to_datetime()
datetime.datetime(3000, 1, 1, 0, 0, 1, 999999, tzinfo=datetime.timezone.utc)   # +1 second
>>> Timestamp.from_datetime(dt.datetime(9999, 12, 31, 23, 59, 59, 999999, tzinfo=dt.timezone.utc))
OverflowError: date value out of range

The Cython packer already uses integer timedelta arithmetic and is correct. This makes the pure-Python from_datetime() do the same, so the two paths agree and the round-trip invariant from_datetime(d).to_datetime() == d holds for far-future datetimes. Naive datetimes keep their existing local-time interpretation (matching datetime.timestamp()).

Follow-up to #662, which fixed the pre-epoch rounding direction in the same method.

Existing tests are unchanged; I added a regression test covering the far-future round-trip, the exact seconds/nanoseconds, agreement with packing the datetime directly, and the former OverflowError case. Full suite is green on both the C-extension and pure-Python paths.

Comment thread msgpack/ext.py Outdated
from_datetime() derived the whole-second part from the float
datetime.timestamp(), which cannot hold microsecond precision far from
the epoch. It rounded the seconds up while still taking the exact
microsecond for the nanoseconds, so the result was one second in the
future (and raised OverflowError near datetime.max).

Use integer timedelta arithmetic, matching the Cython packer, so the
pure-Python path agrees with the reference implementation.
@gaoflow gaoflow force-pushed the fix-from-datetime-far-future-precision branch from 35bf81d to b367f92 Compare July 4, 2026 20:45
@methane methane requested a review from Copilot July 5, 2026 06:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes msgpack.ext.Timestamp.from_datetime() so it computes the whole-second component using integer timedelta arithmetic instead of datetime.timestamp() (float), preventing off-by-one-second rounding and OverflowError for far-future datetimes while keeping naive-datetime “local time” semantics consistent with datetime.timestamp().

Changes:

  • Switch Timestamp.from_datetime() to derive seconds/nanoseconds from dt - epoch using integer math (and handle naive datetimes via astimezone() to preserve existing behavior).
  • Introduce a shared module-level UTC epoch constant and reuse it in to_datetime().
  • Add regression tests covering far-future round-trip precision, exact seconds/nanoseconds, packer agreement, and the prior OverflowError case.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
msgpack/ext.py Reworks Timestamp.from_datetime() to avoid float timestamp precision/overflow and reuses a module-level UTC epoch in datetime conversions.
test/test_timestamp.py Adds regression tests to ensure correct far-future datetime round-trips and consistency with datetime packing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@methane methane merged commit 659ee98 into msgpack:main Jul 5, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants