Skip to content

Avoid encoding context for cached Tabular predictions - #52

Draft
GraysonWu wants to merge 1 commit into
mainfrom
fix/tabular-session-encoding
Draft

GraysonWu wants to merge 1 commit into
mainfrom
fix/tabular-session-encoding

Conversation

@GraysonWu

Copy link
Copy Markdown
Collaborator

Cached Tabular predictions build a full envelope, including encoding every context row, and then discard the context before sending the request. Skip that encoding for session scoring while retaining schema checks, pinned dtype conversion, and query serialization. One-shot predictions and session creation still encode the context.

Adds regression coverage for classification/regression with and without a pinned schema. Public API and all nine wire-payload snapshots remain unchanged.

Validation:

  • Four new cases fail before the fix and pass afterward; focused session/schema tests pass.
  • Live A10G comparison, Large/BF16, one estimator, 10k context rows and 30 numerical features: one-row FittedModel.predict(...).to_pandas() mean fell from 467.4 ms to 52.4 ms over 20 alternating repeats per version. Request payload contents matched (excluding request IDs), and predictions were bitwise identical, at query batch sizes 1, 8 and 1,000. This is a colocated sequential test, not a capacity or P99 claim.
  • Ruff, formatting, mypy, repository hygiene and attribution checks pass.
  • With pandas 3.0.6 and DuckDB 1.5.6 from the lockfile: main suite 2,694 passed / 128 skipped; separate engine suite 2,568 passed / 101 skipped. One existing main-suite contract-provenance check fails because the separately checked-out contract is newer than the generated bindings; it also fails on unchanged main. Latest pandas 3.1 / DuckDB 2.0 additionally expose baseline failures, so those dependencies were not changed in this patch.

Signed-off-by: GraysonWu <graysonw@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant