Skip to content

time with time zone values differ by cursor, and PandasCursor drops the offset #1018

Description

@laughingman7743

Problem

The cursors return different values for an Athena time with time zone column, and PandasCursor drops the offset when it reads the result file from S3.

Measured on master 785be36 (2026-10-03) with SELECT CAST('12:34:56.789 +09:00' AS TIME WITH TIME ZONE) AS t:

Cursor S3 result file Managed query result storage (GetQueryResults)
Cursor '12:34:56.789+09:00' (str) '12:34:56.789+09:00' (str)
PandasCursor datetime.time(12, 34, 56, 789000) (naive, offset dropped) '12:34:56.789+09:00' (str)
ArrowCursor '12:34:56.789+09:00' (str) fails on the plain TIME column of the same query (separate defect)
PolarsCursor '12:34:56.789+09:00' (str) '12:34:56.789+09:00' (str)
  • DefaultTypeConverter and the Arrow and Polars converters map time to _to_time but have no time with time zone entry, so those values stay strings (pyathena/converter.py, pyathena/arrow/converter.py, pyathena/polars/converter.py).
  • AthenaPandasResultSet._PARSE_DATES includes time with time zone, so read_csv parses it as a timezone-aware datetime, and _trunc_date() then takes .dt.time, which keeps the local clock time and drops the +09:00 offset (pyathena/pandas/result_set.py). The same column from GetQueryResults stays a string, so PandasCursor also disagrees with itself across result storage modes.

Expected

One documented representation for time with time zone across cursors and result storage modes, with no silent loss of the offset. Either keep the string everywhere, or convert to a timezone-aware datetime.time everywhere.

Found during the review of #1005 (#935).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions