[alan-kumo]

This PR adds index clocks: a DiskGraph index can store its timestamps in nanoseconds, microseconds, milliseconds, seconds, hours or days. Seconds stays the default, and a seconds index stores exactly the same data as one built on main. The clock changes only the tick that timestamps are stored in and the integer width used to store them; nulls, rounding, truncation, saturation, errors, statistics and prediction output follow main's rules in every clock.

Problem

Teams with event data finer or coarser than one second ask:

• A prediction is anchored at 12:00:00.500. Why can it miss an event from 12:00:00.300, or see one from 12:00:00.700?
• Two clicks arrived 300 ms apart in the same second. Why can sampling not tell which one came last?
• Two anchors 200 ms apart in the same second become training rows with the same anchor. Why do they see the same history, and why can a window not be shorter than a second?
• Our tables only carry dates. Can the index store days instead of seconds?

Cause

Every index stores time as epoch seconds. Ingest converts each source timestamp to whole seconds (typed timestamps and integer epochs round up, text truncates), so everything finer than a second is gone before sampling or training-table generation sees it. DatasetMetadata.timestamp_unit exists, but ingest always writes "seconds" and the sampler asserts it.

<index>/
  index_metadata.json        # "timestamp_unit": "seconds", always
  diskgraph_config.json      # no clock setting
  nodes/<table>/shard-*/     # row timestamps: epoch seconds; edge time keys: 32-bit seconds
  col_stats.json             # timestamp min/max as min_secs / max_secs
  kumo_col_stats/            # kumo_timestamp_YYYY-MM-DD HH:MM:SS

Change

Choose the clock when building a snapshot. The default is seconds.

import kumo.engine as kumo
graph = kumo.Graph(tables=..., edges=...)
snapshot = graph.snapshot(timestamp_precision="milliseconds")

# kumo-studio (hosted)
graph.snapshot(timestamp_precision="milliseconds")  # later snapshot() calls, as in fit, keep milliseconds

The index records its clock, and everything that reads it (sampling, training-table generation, statistics, prediction) works in that clock.

<index>/
  index_metadata.json        # "timestamp_unit": "milliseconds"
  diskgraph_config.json      # "timestamp_unit": "milliseconds"   (always written; absent in older configs means seconds)
  nodes/<table>/shard-*/     # row timestamps in milliseconds; edge time keys 64-bit
  col_stats.json             # min_ms / max_ms + "timestamp_unit"   (seconds keeps min_secs / max_secs)
  kumo_col_stats/            # kumo_timestamp_YYYY-MM-DD HH:MM:SS.fff when a fraction is stored

From a millisecond snapshot:

// index_metadata.json
{ "dataset_name": "<graph id>", "timestamp_unit": "milliseconds", ... }

// col_stats.json, events.TS
{ "min_ms": 1704068319815, "max_ms": 1714435166478, "timestamp_unit": "milliseconds" }

// kumo_col_stats/col_stats_full/events.json
"TS__kumo.timestamp__MAX__False": "kumo_timestamp_2024-04-29 23:59:26.478"

What changes with the clock

Clock	Row timestamp stored as	Edge time key	Range
nanoseconds	ns since 1970	64-bit	1677–2262
microseconds	µs since 1970	64-bit	±292,000 years
milliseconds	ms since 1970	64-bit	±292 million years
seconds (default)	seconds since 1970, as main	32-bit, as main	as main
hours	hours since 1970	configured width (≤32-bit)	very large
days	days since 1970	configured width (≤32-bit)	very large

Row timestamps are variable-length integers, so their size follows the clock (a 2026 timestamp takes 3 bytes in days, 5 in seconds, 9 in nanoseconds). On a 2M-user × 60M-event fixture a milliseconds index is 11.8% larger than a seconds one (2,856 vs 2,554 MB), 240 MB of it from the 64-bit edge time keys and 62 MB from row timestamps; ingest, sampling and training-table times are within noise of seconds.

What stays as on main, in every clock

Value	Rule
Typed timestamps and integer epochs finer than the clock	round up
Text with a fraction	truncate
Decimal epochs with a fraction	truncate to the finer of their unit and the clock, then round up
Date32	whole days; null outside main's accepted range
Unparseable text, unsupported formats	null, as main
Custom training-table timestamps	truncate
Datetime literals in filters	round up; a naive datetime is local time, a pd.Timestamp is UTC
Request and seed anchors	truncate; NodeRange anchors floor
Window and offset lengths	truncate to the clock
Out-of-range edge times	far-future key
Failed temporal validity checks	keep the row
Int64 create/end-time columns	epoch nanoseconds
Training-table and prediction outputs	timestamp[ns] / timestamp[us] exactly as main

Two rules are needed only because a clock can now be too narrow for a value, which never happens in seconds:

• A source value outside the clock's range (for example year 9999 on a nanosecond index) becomes a null cell, and the row is dropped if it is the table's time column, as main does for out-of-range Date32 values.
• A training-table row whose window would run past the clock's range is skipped instead of aborting the job.

Kumo's default prediction anchor comes from the latest timestamp statistic. Statistics now print the stored fraction, and the anchor parser adds the digits beyond microseconds that Python's datetime drops, so on a nanosecond index the default anchor equals the latest event.

Back-compat

Situation	Behaviour
Existing seconds index	Read unchanged.
New seconds index	Index data, index_metadata.json and statistics byte-identical to main. diskgraph_config.json gains one line, "timestamp_unit": "seconds", so its hash in ingest_stats.json changes.
Config or metadata without timestamp_unit	Read as seconds. An index build started by an older version still resumes.
Snapshot cache / hosted snapshot identity, seconds	Unchanged; existing snapshots are reused. Other clocks get their own identity.
Seconds ingest, sampling and training-table generation	Same cost as main.
Plan built for a different clock than the index	Configuration error (DiskGraphPlanCompiler(index_path=...) writes plans in the index clock; every Kumo caller passes it).
New kumo-studio SDK, older server	Seconds works as before; another clock fails clearly instead of silently building seconds.
Older SDK, new server	Builds seconds, as before.
Hosted workflows in flight	The clock is passed to the ingest child only behind the diskgraph-index-clock-v1 patch.
Prediction Parquet	Unchanged: timestamp[us].

Known limitations

• Changing a snapshot's clock means building a new index.
• Sub-second clocks use 64-bit edge time keys, so they cannot share a packed key with an edge weight (Kumo never sets one); this is a configuration error.
• A window shorter than one tick is empty (for example 30 minutes on an hours index), because window lengths truncate to the clock as on main.
• Exported prediction anchors are timestamp[us], so a nanosecond index's exported anchors are truncated to the microsecond.

Testing Done

• Seconds equals main. An index built from a fixture with every source type (fractional text with and without formats, s/ms/µs/ns epochs, decimals, Date32 beyond 2038, typed timestamps, nulls, time-keyed edges) is byte-identical to main's apart from the added timestamp_unit line in diskgraph_config.json (and its hash) and files that also differ between two runs of main. Sampled subgraphs and every training-table file match.
• Every clock gives the same results on whole-day data (decoded values, samples, training tables, statistics), and follows main's null and rounding rule per source type on messy data.
• Speed. Seconds-clock ingest encode against main, medians of 9 interleaved runs on 5M rows: text +0.2%, decimal −3.3%, Arrow −1.7%, integer epochs +3.3% wall time at equal CPU time. Training-table generation on 2M users × 60M events could not be separated from noise on a shared machine: across three interleaved runs, day windows were 4–13% faster than main and month windows 0–12% slower.
• End to end. For all six clocks through kumo-engine: snapshot, training table, fit, predict and export complete; exported timestamps are timestamp[us]; the seconds snapshot id equals main's.
• Suites, on the final head. cargo clippy --all-targets -D warnings clean for diskgraph-core, -rowpack, -ingest and -pybind; cargo test --release for core, rowpack and ingest 1212 passed, plus one disk-pressure test that measures free space on a busy shared filesystem and passes on a quiet one; test_index_clocks.py 17 passed; kumoai_studio 654 passed; kumo-core runtime and pquery with kumo-api 1285 passed; kumo-engine snapshot and graph tests 30 passed; hosted snapshot, data-backend, refresh-workflow, connector, datamodule and link-prediction module tests 403 passed. The remaining kumoai_studio failures come from an incompatible local pyarrow and an auto-init check.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

• Last reviewed commit: 16ad4d7
• Latest commit, not reviewed: f42fb95

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

The changes add six index timestamp clocks: seconds, nanoseconds, microseconds, milliseconds, hours, and days. Seconds remain the default. Timestamp conversion, parsing, rowpack storage, sort keys, sampling, temporal validation, plan execution, statistics, and output now use the selected clock. Ingest, snapshot, and plan APIs carry the clock into index construction. Extensions inherit the parent clock. Compatibility handling and tests cover conversion, saturation, metadata, and timestamp output.

Merge Risk: Moderate · previous review 16ad4

An unsupported timestamp sort key can produce an index with incorrect ordering without failing ingest. Other outstanding timestamp-output and diagnostic concerns remain. Resolve or explicitly accept these risks before merging.

Pre-merge checks: 1 · 1 · 1

Failed checks: 1 warning · 1 inconclusive

Check name	Status	Explanation	Resolution
Pr Size		The authoritative review-scoped diff changes 121 files. This exceeds the 40-file limit.	Split the pull request into smaller, reviewable units.
Change Map Design Review		The required <!-- kumo-pr-change-map --> conversation comment is not included in the supplied review context, and no change-map comment exists in the checkout. The PR description only links to an ex…	Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including its linked problem, per-change rationale, abstraction-layer decisions, existing-behavior analysis, focused verification, and stopping condi…

Passed checks: 1 passed

Check name	Status	Explanation
Public Resource Access		No added or modified external URL was found in the pull-request diff. The only added package source reference is the relative path ../diskgraph-core in `packages/diskgraph/diskgraph-rowpack/Cargo.to…

Full details: Public Resource Access

Explanation

No added or modified external URL was found in the pull-request diff. The only added package source reference is the relative path ../diskgraph-core in packages/diskgraph/diskgraph-rowpack/Cargo.toml:12; its target exists in the same public repository at packages/diskgraph/diskgraph-core. No inaccessible resource reference was introduced.

Full details: Change Map Design Review

Explanation

The required <!-- kumo-pr-change-map --> conversation comment is not included in the supplied review context, and no change-map comment exists in the checkout. The PR description only links to an external maintained map. The repository guidance confirms that the map contains the rationale and file-level design details that this check must assess. The code diff is available, but it cannot establish whether each change was necessary, correctly layered, minimally scoped, and supported by the required verification and stopping condition without that comment.

Resolution

Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including its linked problem, per-change rationale, abstraction-layer decisions, existing-behavior analysis, focused verification, and stopping condition. Then reassess the design against the reviewed diff.

Finishing Touches: 💡 1

⚔️ Resolve merge conflicts: 💡

• Resolve merge conflict in branch fix/native-subsecond-timestamps

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[alan-kumo]

Change map

File	What changed	Why
packages/diskgraph/diskgraph-core/src/time.rs	Microsecond, millisecond, hour and day unit systems; one clock list and with_index_clock! dispatch; round-up, floor and truncate conversions with clock-constant factors; calendar and offset arithmetic return None past the clock's range	The six index clocks and main's rounding rules in each, without panics
packages/diskgraph/diskgraph-core/src/sortkey.rs packages/diskgraph/diskgraph-core/src/metadata.rs	64-bit timestamp sort-key kind; DatasetMetadata.timestamp_unit read with a seconds default	Sub-second clocks need 64-bit edge time keys; old indexes read as seconds
packages/diskgraph/diskgraph-core/src/tt_builder.rs	Plan carries its clock; Op::map_clock converts a target op's times into another clock	The builder converts plan times once, at build()
packages/diskgraph/diskgraph-rowpack/src/access.rs packages/diskgraph/diskgraph-rowpack/src/parser.rs packages/diskgraph/diskgraph-rowpack/src/lib.rs packages/diskgraph/diskgraph-rowpack/Cargo.toml packages/diskgraph/Cargo.lock packages/diskgraph/third_party/artifact-manifest.json	Row reader carries the index clock; main's parsing and null rules kept; only the final conversion targets the clock; decimal epochs keep their fraction down to the tick; values outside the clock's range read as null; decoders emit the clock's Arrow unit; rowpack depends on diskgraph-core for TimeUnit (lockfile and artifact manifest regenerated)	Store row timestamps in the index clock; seconds stays byte-identical to main
packages/diskgraph/diskgraph-rowpack/src/tests_int_string_list.rs	Test row reader built with the clock	Constructor signature
packages/diskgraph/diskgraph-ingest/src/graph_config.rs packages/diskgraph/diskgraph-ingest/src/lib.rs packages/diskgraph/diskgraph-ingest/src/stage_extension.rs	Optional timestamp_unit (written for every clock, read as seconds when absent); recorded in index_metadata.json; edge time-key width per clock; far-future clamp kept; stage extensions inherit the parent clock; Encode markers without the field still resume	Ingest into the configured clock with backward-compatible configs
packages/diskgraph/diskgraph-ingest/src/col_stats.rs	Timestamp statistics in clock ticks; min_secs/max_secs kept for seconds, min_<unit>/max_<unit> + timestamp_unit otherwise; Kumo timestamp strings print a stored fraction	Statistics in the index clock; seconds output unchanged
packages/diskgraph/diskgraph-ingest/src/config_converter/convert.rs packages/diskgraph/diskgraph-ingest/src/config_converter/inference.rs packages/diskgraph/diskgraph-infer/src/lib.rs	Pass the clock through config conversion and temporal inference	Callers construct configs and readers with the clock
packages/diskgraph/diskgraph-ingest/README.md	Short section on index clocks	User-facing documentation
packages/diskgraph/diskgraph-ingest/src/config_converter/tests.rs packages/diskgraph/diskgraph-ingest/src/tests_unit.rs packages/diskgraph/diskgraph-ingest/tests/graph_config.rs packages/diskgraph/diskgraph-ingest/tests/resolution_ingest.rs packages/diskgraph/diskgraph-ingest/tests/resolution_runtime_batch.rs packages/diskgraph/diskgraph-ingest/tests/seed_table_ingest.rs	Existing tests set the new config field; new tests for key widths, far-future keys, configs without a clock	GraphConfig has no Default; cover the new behaviour
packages/diskgraph/diskgraph-index/src/graph.rs packages/diskgraph/diskgraph-index/src/bin/explore.rs	Temporal validity and decoding in the index clock with main's rules (Int64 = epoch ns, Date32 = days, failed check keeps the row)	Index reads follow the clock
packages/diskgraph/diskgraph-engine/src/engine.rs	Engine reads the clock from metadata; seed timestamps truncate and NodeRange anchors floor into it	Sampling in the index clock
packages/diskgraph/diskgraph-train-table/src/ttgen_executor.rs packages/diskgraph/diskgraph-train-table/src/ttgen_resolve.rs packages/diskgraph/diskgraph-train-table/src/ttgen_traversal.rs packages/diskgraph/diskgraph-train-table/src/ttgen_output.rs packages/diskgraph/diskgraph-train-table/src/ttgen_stats.rs packages/diskgraph/diskgraph-train-table/src/split_decider.rs packages/diskgraph/diskgraph-train-table/src/custom_tt_chunk_processor.rs	Remove hard-coded seconds from generic code; windows, splits and outputs in the clock; anchors whose windows leave the clock are skipped; custom-table times truncate; outputs saturate; statistics fall back instead of panicking	Training tables in the index clock with main's rules
packages/diskgraph/diskgraph-train-table/tests/custom_tt_integration.rs packages/diskgraph/diskgraph-train-table/tests/ttgen_e2e.rs packages/diskgraph/diskgraph-train-table/tests/ttgen_integration.rs	Tests pass the clock; skipped-row and calendar-limit cases	Signature changes and new behaviour
packages/diskgraph/diskgraph-pybind/src/tt_executor.rs packages/diskgraph/diskgraph-pybind/src/custom_tt_entry.rs packages/diskgraph/diskgraph-pybind/src/tt_builder.rs packages/diskgraph/diskgraph-pybind/src/lib.rs	Dispatch once on the index clock; plan clock must match the index; builder keeps times as given and converts at build(); exact datetime literals rounded up; Python TimeUnit gains the new clocks	Bindings for the clocks
packages/diskgraph/diskgraph-pybind/src/temporal.rs packages/diskgraph/diskgraph-pybind/src/bp_col_stats.rs	resolve_temporal(..., timestamp_unit=) and batch-prediction statistics take the clock (default seconds)	Kumo statistics in the snapshot's clock
packages/diskgraph/diskgraph-pybind/src/tt_executor/tests.rs packages/diskgraph/diskgraph-pybind/src/tt_executor/pseudo_time_tests.rs	Tests pass the clock	Signature changes
packages/diskgraph/diskgraph-pybind/python/diskgraph/compiler.py packages/diskgraph/diskgraph-pybind/python/diskgraph/tt_artifacts.py	DiskGraphPlanCompiler(index_path=) writes plans in the index clock	Plans match the index
packages/diskgraph/diskgraph-pybind/python/tests/test_index_clocks.py	Every clock gives the seconds result on whole-day data and main's rules on messy data	Pins the invariant
packages/kumo-api/kumoapi/typing.py	TimestampPrecision and validate_timestamp_precision	One list of clock names
packages/kumo-core/src/kumo_core/diskgraph/ingest.py	DiskgraphIngestSpec.timestamp_precision written into the ingest config's timestamp_unit when not seconds	Pass the clock to ingest
packages/kumo-core/src/kumo_core/pquery.py packages/kumo-core/src/kumo_core/prediction.py packages/kumo-core/src/kumo_core/training/table.py packages/kumo-core/src/kumo_core/diskgraph/lightning_data.py packages/kumo-core/src/kumo_core/diskgraph/runtime_adapter.py packages/kumo-core/src/kumo_core/train/shallow_rhs_link_gnn_module.py	Plans compiled with index_path; hours and days in the seed/NodeRange unit maps; timestamp statistics keep every nanosecond digit; a temporal-LP training batch takes its valid RHS items at its earliest anchor	Training and prediction follow the index clock
packages/kumo-core/tests/test_diskgraph_runtime.py packages/kumo-core/tests/test_pquery.py packages/kumo-core/tests/test_prediction.py packages/kumo-core/tests/test_train_table.py	Tests for the above, including exact anchor parsing across ISO formats	Coverage
packages/kumo-engine/kumo/engine/graph.py	Graph.snapshot(timestamp_precision=); seconds keeps the existing cache key	User-facing option in the pip SDK
packages/kumo-engine/test/test_snapshot_clock.py	Six clocks accepted; seconds cache key unchanged; one snapshot per clock	Coverage
packages/kumo-sdk/kumoai_studio/graph/graph.py packages/kumo-sdk/kumoai_studio/client/graph.py packages/kumo-sdk/kumoai_studio/backend/base.py	Graph.snapshot(timestamp_precision=), which without a precision keeps the clock of the graph's current snapshot; query parameter only when not seconds; checks the server's clock echo	User-facing option in the hosted SDK
packages/kumo-sdk/test/graph/test_snapshot_clock.py	Echo matrix for old and new servers; a snapshot without a precision reuses the current clock	Coverage
rest/public/routers/data_snapshot.py rest/driver/data_snapshot_api_driver.py rest/config/data_backend.py rest/config/graph_v2.py	Snapshot request accepts the clock; driver records it; router echoes the stored clock in X-Kumo-Timestamp-Precision	Hosted snapshots in the requested clock
rest/backends/data_backend/data_backend.py rest/backends/data_backend/activities/data_backend_activities.py rest/backends/data_backend/workflows/graph_refresh_workflow.py rest/backends/data_backend/workflows/diskgraph_ingest_workflow.py pool/activities.py pool/diskgraph_args.py	Clock carried through version info, ingest args and the refresh workflow (behind the diskgraph-index-clock-v1 patch); seconds snapshot identity unchanged; prediction statistics use the recorded clock	Hosted ingest and statistics
kumo/diskgraph/ingest.py kumo/diskgraph/service.py kumo/data/generic/diskgraph_batch_pred.py kumo/data/generic/diskgraph_train_table.py	Ingest config gets the clock; hosted plans compiled with index_path; statistics clock passed through	Hosted workers
kumo/connector/timestamp_convert.py kumo/connector/utils.py kumo/connector/connector.py	One vectorized conversion into any clock (timestamp_ticks_from_array); text statistics truncate into the snapshot clock	Connector statistics in the snapshot's clock
test/rest/backends/test_snapshot_clock.py test/rest/backends/test_data_backend.py test/rest/backends/test_diskgraph_graph_refresh_workflow.py test/connector/local/test_timestamp_convert.py	Hosted identity, workflow argument, echo and statistics tests	Coverage
test/data/generic/test_diskgraph_batch_pred.py test/data/generic/test_diskgraph_generate_train_table.py test/data/generic/test_diskgraph_predict_plan_ttng.py test/data/generic/test_diskgraph_runtime_adapter.py test/data/generic/test_diskgraph_train_table_forecasting.py test/data/generic/test_ttng_cross_graph_contamination_e2e.py test/train/datamodule/test_diskgraph_lightning_data.py test/train/test_shallow_rhs_link_gnn_module.py	Test doubles accept index_path; fixtures write index_metadata.json; hours/days seed conversion; earliest-anchor valid RHS items	Signature changes and coverage

[alan-kumo]

@coderabbitai review

Please review the native timestamp precision changes and their compatibility with existing second-based indexes. Focused native and SDK integration tests pass; the broader Rust suites are still completing.

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 6

---

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/kumo-engine
• Branch: main
• File: .coderabbit.yaml
• Commit: 3961984
• Layers: Repository YAML
• Review profile: QUIET
• Plan: Enterprise
• Run ID: 12758629-93fe-40a9-a520-9ab9213696c5

Commits

Reviewing files that changed from the base of the PR and between 3961984 and 8053470.

Files selected for processing: 44

• packages/diskgraph/diskgraph-core/src/metadata.rs
• packages/diskgraph/diskgraph-core/src/sortkey.rs
• packages/diskgraph/diskgraph-core/src/time.rs
• packages/diskgraph/diskgraph-engine/src/engine.rs
• packages/diskgraph/diskgraph-index/src/bin/explore.rs
• packages/diskgraph/diskgraph-index/src/graph.rs
• packages/diskgraph/diskgraph-ingest/README.md
• packages/diskgraph/diskgraph-ingest/src/col_stats.rs
• packages/diskgraph/diskgraph-ingest/src/lib.rs
• packages/diskgraph/diskgraph-ingest/src/stage_extension.rs
• packages/diskgraph/diskgraph-pybind/python/diskgraph/compiler.py
• packages/diskgraph/diskgraph-pybind/python/diskgraph/tt_artifacts.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_sampling.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_subsecond_time.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_tt_artifacts_split_marshalling.py
• packages/diskgraph/diskgraph-pybind/src/custom_tt_entry.rs
• packages/diskgraph/diskgraph-pybind/src/tt_builder.rs
• packages/diskgraph/diskgraph-pybind/src/tt_executor.rs
• packages/diskgraph/diskgraph-rowpack/src/access.rs
• packages/diskgraph/diskgraph-rowpack/src/bin/rowpack_ctx_inspect.rs
• packages/diskgraph/diskgraph-rowpack/src/lib.rs
• packages/diskgraph/diskgraph-rowpack/src/parser.rs
• packages/diskgraph/diskgraph-rowpack/src/tests_timestamp_precision.rs
• packages/diskgraph/diskgraph-train-table/README.md
• packages/diskgraph/diskgraph-train-table/src/custom_tt_chunk_processor.rs
• packages/diskgraph/diskgraph-train-table/src/pass_b.rs
• packages/diskgraph/diskgraph-train-table/src/route.rs
• packages/diskgraph/diskgraph-train-table/src/split_decider.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_executor.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_output.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_resolve.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_stats.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_traversal.rs
• packages/kumo-core/src/kumo_core/diskgraph/ingest.py
• packages/kumo-core/src/kumo_core/prediction.py
• packages/kumo-core/src/kumo_core/train/diskgraph_nested_trainer.py
• packages/kumo-core/src/kumo_core/train/io.py
• packages/kumo-core/src/kumo_core/train/output.py
• packages/kumo-core/src/kumo_core/training/table.py
• packages/kumo-core/tests/test_prediction.py
• packages/kumo-core/tests/test_train_output.py
• packages/kumo-core/tests/test_train_table.py
• packages/kumo-engine/kumo/engine/graph.py
• packages/kumo-engine/test/test_prediction_output_columns.py

Files with no reviewable changes: 1

• packages/kumo-core/src/kumo_core/train/output.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[alan-kumo]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 1

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Other comments: 1

packages/diskgraph/diskgraph-rowpack/src/access.rs: 1

2504-2511: Minor · Functional Correctness · Quick win

Date32 range check is lost for Seconds precision.

get_timestamp rejects Date32 days outside -106752..=24855. The reason in its comment is that seconds-based sort keys use i32 epoch seconds.

get_timestamp_with_precision now handles Date32 before it delegates to get_timestamp. For Seconds it returns days * 86_400 with only an overflow check. A date after 2038 now produces a value above i32::MAX. encode_timestamp_value clamps it into the last Int32 bucket, so temporal ordering is silently wrong on legacy seconds indexes.

Apply the legacy range check in the Seconds arm. Only the nanosecond arm needs the unrestricted range.

Proposed fix

         if matches!(array.data_type(), DataType::Date32) {
             let days = array.as_any().downcast_ref::<arrow_array::Date32Array>().ok_or_else(|| anyhow!("date cast"))?.value(self.row);
+            if precision == crate::TimestampPrecision::Seconds && !(-106752..=24855).contains(&days) {
+                return Err(anyhow!("date out of range: day {days}"));
+            }
             let factor = match precision {

---

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/kumo-engine
• Branch: main
• File: .coderabbit.yaml
• Commit: 3961984
• Layers: Repository YAML
• Review profile: QUIET
• Plan: Enterprise
• Run ID: 554a1774-853a-4d2c-a41d-1273cfede30f

Commits

Reviewing files that changed from the base of the PR and between 8053470 and 0f35e0e.

Files selected for processing: 24

• packages/diskgraph/diskgraph-core/src/sortkey.rs
• packages/diskgraph/diskgraph-core/src/tt_builder.rs
• packages/diskgraph/diskgraph-index/src/graph.rs
• packages/diskgraph/diskgraph-ingest/src/lib.rs
• packages/diskgraph/diskgraph-pybind/python/diskgraph/compiler.py
• packages/diskgraph/diskgraph-pybind/python/diskgraph/tt_artifacts.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_compiler.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_subsecond_time.py
• packages/diskgraph/diskgraph-pybind/python/tests/test_tt_artifacts_split_marshalling.py
• packages/diskgraph/diskgraph-pybind/src/tt_builder.rs
• packages/diskgraph/diskgraph-rowpack/src/access.rs
• packages/diskgraph/diskgraph-rowpack/src/parser.rs
• packages/diskgraph/diskgraph-rowpack/src/tests_timestamp_precision.rs
• packages/diskgraph/diskgraph-train-table/README.md
• packages/diskgraph/diskgraph-train-table/src/custom_tt_chunk_processor.rs
• packages/diskgraph/diskgraph-train-table/src/pseudo_time.rs
• packages/diskgraph/diskgraph-train-table/src/split_decider.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_accumulator.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_executor.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_output.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_resolve.rs
• packages/diskgraph/diskgraph-train-table/src/ttgen_traversal.rs
• packages/diskgraph/diskgraph-train-table/tests/custom_tt_integration.rs
• packages/diskgraph/diskgraph-train-table/tests/ttgen_integration.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[alan-kumo]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 1

---

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/kumo-engine
• Branch: main
• File: .coderabbit.yaml
• Commit: 3961984
• Layers: Repository YAML
• Review profile: QUIET
• Plan: Enterprise
• Run ID: 8b570156-4574-40db-9c46-1d5ca746fdb8

Commits

Reviewing files that changed from the base of the PR and between 0f35e0e and c2032ca.

Files selected for processing: 3

• packages/diskgraph/diskgraph-ingest/src/lib.rs
• packages/diskgraph/diskgraph-rowpack/src/access.rs
• packages/diskgraph/diskgraph-rowpack/src/tests_timestamp_precision.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

[alan-kumo]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Major · Validate timestamp precision before changing the output. · lib.rs:7114

packages/diskgraph/diskgraph-ingest/src/lib.rs:7114
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Validate timestamp precision before changing the output.

A fresh ingest removes the local publication marker and deletes the output root before IngestContext::new validates timestamp precision. A precision mismatch can therefore delete an existing index without producing a replacement. Move this validation before invalidate_publication() and the fresh-output wipe. Preserve the existing Encode-marker identity when moving the validation.

---

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/kumo-engine
• Branch: main
• File: .coderabbit.yaml
• Commit: 3961984
• Layers: Repository YAML
• Review profile: QUIET
• Plan: Enterprise
• Run ID: 2f3a3fd9-67b8-49d5-b7e3-30aa0bd92b05

Commits

Reviewing files that changed from the base of the PR and between c2032ca and dbc4c42.

Files selected for processing: 5

• packages/diskgraph/diskgraph-ingest/src/lib.rs
• packages/diskgraph/diskgraph-pybind/python/tests/test_subsecond_time.py
• packages/diskgraph/diskgraph-rowpack/src/access.rs
• packages/diskgraph/diskgraph-rowpack/src/lib.rs
• packages/diskgraph/diskgraph-rowpack/src/tests_timestamp_precision.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[alan-kumo]

@coderabbitai review

The outside-diff finding about validation order is fixed in a4ee0f3. Precision normalization now runs before publication invalidation, output preparation, or a fresh-output wipe. Encode marker bytes are captured first, preserving the prior identity contract. A native regression asserts that an invalid precision configuration preserves every byte of an existing index, including its publication marker. Focused Rust checks pass; native end-to-end and resume-equivalence checks are running.

Please check the complete precision-validation and publication path for any remaining issues.

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[alan-kumo]

@coderabbitai review

Commit aed86e2 fixes the hosted native CI findings. The observed-encoding regression now supplies one key buffer for its one-column context, satisfying the debug-build invariant; all five timestamp regressions pass with debug assertions enabled. The timestamp reader uses ? for the Int64 downcast, satisfying Clippy while preserving its existing Option behavior. Hosted workspace tests and Clippy are rerunning. The separate cloud-backed import job fails during AWS credential configuration before tests execute.

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[alan-kumo]

Commit 6b5723d uses expect_err directly in the ordinary timestamp-overflow regression, resolving the remaining Clippy finding. The focused regression passes. Hosted cargo test --locked --workspace passed on aed86e2; native CI is rerunning on this test-only follow-up.

[alan-kumo]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[aaron-kumo]

Vedaant and I talked this through offline. Here's where we landed, followed by the changes I'm requesting.

Seconds stays the default, and finer precision is opt-in. Warehouse timestamps usually carry sub-second digits whether or not those digits mean anything, for example when an ETL job stamps rows with clock.now(). For most models, one second is as fine a time resolution as is useful. Finer precision also costs more than it looks, because a snapshot stores each timestamp several times: once in the row store, and again in the sort key of every time-keyed edge direction (see the cost section at the end). Detecting precision from the values doesn't get around this. Nearly every column would look fractional and opt itself in. So I agree with a seconds default and an explicit opt-in for data where order within a second matters.

The comments below cover three things: closing the same-second leak in the default clock, which this PR doesn't do; making the opt-in deliver what the description promises; and the shape of the API.

Correctness of the default seconds clock

Both of these are bugs on main today and don't depend on clocks, so they could land first as their own PR.

1. Round consistently: events up, anchors down. An event is visible when event <= anchor. If events round up and anchors round down, a rounded comparison can hide an event but can never reveal a future one. The cost is hiding events from earlier in the anchor's own second, which is the safe side. Typed timestamps and integer epochs already round up, but text and decimal values round down in every clock. A text event at 12:00:00.700 is stored as 12:00:00, so a 12:00:00 anchor sees it, and it lands in the input history instead of the label window. PQL-generated anchors are whole seconds, so this affects ordinary training on any text or decimal timestamp column with fractions, not just custom anchors. This is the leak behind the PR's second motivating question, and the PR keeps it in every clock. Separately, anchors are truncated toward zero, which rounds pre-1970 anchors up; they should be floored. Fixing text and decimal rounding changes stored bytes versus main for fractional values, and that's intended.

2. Carry the original anchor through to output. Each anchor is converted to the clock, and the converted value is what gets written out. Two anchors for the same entity in the same second (say .2 and .8) therefore produce rows that can't be told apart or joined back to the request. That's the other half of the PR's second question, and the PR only solves it if the user opts into a finer clock. Instead, pass the original anchor through training-table generation and prediction unchanged and write it to the output. Use the rounded value only for comparisons and split hashing. This only matters for user-supplied anchors (custom training tables, prediction seeds), since PQL-generated anchors are already whole seconds.

Correctness when someone opts in

3. Grouped link-prediction batches can leak on a sub-second clock. For temporal link prediction, training seeds are grouped by timestamp so that each batch shares one set of valid RHS candidates. That set is computed at the first seed's time, and the buckets are one second wide in every clock. On a seconds index, every seed in a bucket has the same time, so this is fine. On a finer clock, seeds in the same bucket have different times and aren't sorted by time. The batch time can then be later than some seeds' own times, and those seeds get RHS candidates that didn't exist yet at their anchor. The simplest fix is to use the batch's earliest seed time. That can't leak, and it only hides candidates created within that second. Bucketing on the exact clock value would be exact, but it produces many small buckets.

4. Decimal epochs ignore the opt-in. The decimal's fraction is dropped in the source unit before conversion to the clock, so 1700000000.123 seconds becomes 1700000000 even on a millisecond clock. Scale the value to the clock's tick first, then round up as in 1. Decimal epochs are common; for example, Snowflake integer epochs can arrive as parquet decimals.

5. The model still sees whole seconds. The relative-time encoder gets seed and neighbor times integer-divided to seconds, so two clicks 300 ms apart still get the same time feature. That's the PR's first motivating question. Opting in changes which neighbors are visible and in what order, but not the model's sense of time. Either compute time differences at full resolution and scale them to float seconds, or document that opting in only affects ordering and cutoffs.

6. Export truncates to microseconds. Prediction export casts timestamps to µs without checking for lost precision, so nanosecond anchors are silently truncated. Once 2 passes original anchors through, anchors less than 1 µs apart collide again. The description lists this as a known limitation. Dropping nanoseconds (8) removes it; otherwise, export at the index's precision.

API and ergonomics

7. Make the opt-in a per-column setting, not a Graph or snapshot() parameter. The setting controls how a column's values are converted at ingest. That's the same kind of rule as timestamp_format, so it belongs on the column. A per-column setting also fixes two practical problems:

   • Implicit snapshots honor it. In kumo-studio, fit and predict call graph.snapshot() without a unit. snapshot() defaults to seconds and rebuilds whenever the cached snapshot's unit differs. Today, a user who built a millisecond snapshot gets a full re-ingest into a new seconds snapshot as soon as they call fit, without any warning. Column settings are saved with the table definitions, so every snapshot sees them, whether it's built implicitly or by a later refresh. Studio then doesn't need to require explicit snapshots.
   • The cost follows the opt-in. Each edge time key comes from one table's time column, so a per-column setting maps directly onto per-relation key widths.

   This PR doesn't need per-relation key widths. The index can use the finest resolution any column declares and round undeclared columns up to whole seconds. Each column then gets exactly the behavior it declared; only the storage cost is index-wide for now, and per-relation widths can reduce it later without an API change. Drop the Graph and snapshot() parameters, along with the REST parameter and header, so the setting lives in one place.

   On naming: "unit" is taken. TimestampUnit (accepted by timestamp_format) and the internal per-column timestamp_unit already mean the source epoch unit, so the PR's graph-level timestamp_unit reuses the name for a different concept. Since timestamp_format already describes the source, the new name should describe what Kumo does to the values. Three options:

   • keep_subsecond: bool = False. It can't be mistaken for a description of the source, and the default is obvious. This fits if the only finer option is microseconds, which matches export (6) and costs about the same as milliseconds: both use 64-bit edge keys and differ by about 2 bytes per row value.
   • timestamp_rounding: "second" | "millisecond" | "microsecond". It says this is a transformation Kumo applies, in the same vocabulary as pandas' .round('s'), and it extends to several units. The docs should say values are rounded up.
   • timestamp_resolution. It describes the effect accurately, but next to timestamp_format it reads like a second description of the source.

   I lean slightly toward keep_subsecond if microseconds is the only finer option, and timestamp_rounding otherwise. Either way, the opt-in rarely needs timestamp_format as well; a typed TIMESTAMP(9) column needs no format. A column declared as integer epoch seconds that opts in has no fractions to keep, so it deserves at least a warning.

8. Offer fewer units.

   • Drop hours and days. The day clock keeps 32-bit edge keys, so it saves only about 2 bytes per row timestamp. In exchange, the whole index loses time of day, and any window shorter than a day silently comes back empty. Date-only columns are already exact in a seconds index. PQL's finest window unit is seconds, so dropping these two clocks also gets rid of the "window shorter than one tick" case. If they stay, reject such windows instead of returning empty results.
   • Reconsider nanoseconds. Int64 nanoseconds only cover the years 1677–2262, so common sentinel dates (0001-01-01, 9999-12-31) saturate, and export is µs anyway. Milliseconds and microseconds probably cover the real use cases.

Description and claims

9. Please attach the proofs and the fuzz harness. The description cites Kani and z3 proofs and a 1.5M-string text fuzz, but none of it is in the diff, so there's nothing to review. The z3 claim ("data rounding up … no future leakage in every clock") also doesn't match the code: text and decimal values round down (1), so whatever was proven doesn't model those paths. The fuzz compares against main's parser, so it shows that this PR matches main, truncation included. It doesn't show that either is correct.

Other things to consider

• When to opt in. The docs should say when it's worth it: order within a second carries signal (clickstreams, trading), and the timestamps are real event times rather than ETL load times. A warehouse type like TIMESTAMP(9) on its own isn't a reason to opt in.
• Discoverability (nice-to-have). Nearly every column has fractions, so a "fractions detected" hint would just be noise. A more useful signal is a high rate of same-entity events landing in the same second, which suggests ordering information is being lost. Ingest could report that.

The cost of opting in, and an idea for reducing it

A snapshot stores each timestamp more than once on purpose. Each edge's sort key carries the neighbor's timestamp, so the sampler can find the part of an adjacency list that's visible at an anchor by binary search, without reading every neighbor's row. The row store keeps its own copy for features and training-table generation. The total is one row copy plus one copy per time-keyed edge direction. Every foreign key on an event table adds a reverse direction keyed by the event's time, and forward directions into tables with their own time column add more. An event table with three foreign keys stores each timestamp at least four times.

When a user opts in, the row copy is a varint and grows by only 1–3 bytes, but every edge-key copy doubles from 4 to 8 bytes. Edge keys therefore dominate the opt-in cost, and the PR only benchmarks the seconds clock. Before we settle the design, it would be worth measuring index size and sampling throughput for seconds vs. milliseconds on a representative dataset.

If the 64-bit key cost turns out to be large, one option is to keep 32-bit seconds keys even in an opted-in index and recover exactness elsewhere:

• Ordering: at ingest, sort edges by (seconds key, exact time), so the order within each second is exact even though the key is coarse.
• Cutoff: binary-search on seconds as today, then compare exact times only for neighbors in the anchor's own second, reading them from the row store.

That keeps edge keys at today's size. The cost is extra row reads for neighbors in the boundary second. That's cheap for typical data but expensive for the dense clickstream data most likely to opt in, so it's only worth doing if the measured key cost is large. I'm not suggesting it for this PR.