Skip to content

feat: retry locked loads, read env vars, clear workspace error - #219

Merged
eddietejeda merged 5 commits into
mainfrom
feat/load-retry-and-env
Oct 15, 2026
Merged

eddietejeda merged 5 commits into
mainfrom
feat/load-retry-and-env

Conversation

@eddietejeda

Copy link
Copy Markdown
Contributor

Closes the SDK gaps reported by Tenki. Five commits, one per item; every hand-written piece is generator-ignored or re-applied by a patch script from regenerate.yml, so a regen keeps it.

1. Retry locked loads (feat(loads))

The server serialises loads per table; a second writer gets 409 {"error":{"code":"RESOURCE_LOCKED",...}} + Retry-After. New hotdata/loads.py subclasses the generated DatabasesApi (load_database_table) and ConnectionsApi (load_managed_table, the only other /loads operation) and retries on 409 + code RESOURCE_LOCKED only, matched on the code (from exc.data or the raw JSON body), never the message. Retry-After is honoured (uncapped, jitter on top), else exponential backoff; LoadRetryPolicy defaults to a 60 s budget / 10 retries; exhaustion raises TableLockedError (a 409 ApiException with attempts and the last response). CONFLICT / IDEMPOTENCY_KEY_CONFLICT / other errors pass through untouched. Exported as hotdata.DatabasesApi / ConnectionsApi via patch_query_exports.py, same swap as QueryApi.

2. Environment variables (feat(config))

Configuration() defaults api_key from HOTDATA_API_KEY, workspace_id from HOTDATA_WORKSPACE and host from HOTDATA_API_URL when the argument is None, matching the CLI. Explicit args win; an empty variable is unset. configuration.py is generated from the in-repo configuration.mustache, so the change is in the template too (a test pins that).

3. Clear no-workspace error (feat(client))

ApiClient.update_params_for_auth now calls hotdata._workspace.ensure_workspace: if the operation's security lists WorkspaceId and no value would be sent, it raises hotdata.MissingWorkspaceError (an ApiValueError) before the request goes out, naming workspace_id=, HOTDATA_WORKSPACE and the X-Workspace-Id header. Endpoints with BearerAuth only (the Workspaces endpoints, per the spec) are unaffected, as are calls passing the header via _headers or overriding auth via _request_auth. The spec has no session scheme any more, so there is nothing else to accept. Applied by new scripts/patch_workspace_guard.py (wired into the workflow).

4. wait_for_result docs + fetch_ready_result (feat(query))

wait_for_result's docstring now states up front that it polls with limit=0 and returns a status-only response with no rows. New QueryApi.fetch_ready_result(result_id, x_database_id, offset=, limit=, format=) does the same wait and then one get_result for the rows. Existing behaviour unchanged.

5. Reject unknown model fields (feat(models)!, breaking)

scripts/patch_model_extra_forbid.py adds extra="forbid" to every model used as a request body plus the models nested in them (29 models; the ColumnDefinition oneOf wrapper has no plain fields and is skipped). Unknown kwargs now raise pydantic.ValidationError. Response models are deliberately left alone so a newer server never breaks an older SDK; the existing forward-compat test still passes. The whole suite, the SDK's own code and the integration tests (checked statically) pass no unknown fields. Changelog entry is marked Breaking under [Unreleased] ### Changed.

Tests

  • tests/test_loads.py (16): retry/Retry-After/backoff/exhaustion, code-not-message matching, other 409s and non-409s untouched, arg forwarding, export wiring.
  • tests/test_config_env.py (7): env defaults, precedence, empty-is-unset, wire check, template parity.
  • tests/test_workspace_guard.py (10): fails before sending, env/arg/header/_request_auth paths, Workspaces endpoints still work, workflow wiring.
  • tests/test_query.py (+4): status-only contract, fetch_ready_result wait-then-fetch, paging args, failure propagation.
  • tests/test_model_extra_forbid.py (6): unknown kwarg raises, aliases still work, responses still ignore extras, closure coverage, idempotency + workflow wiring.

pytest test tests --ignore=tests/integration: 422 passed. mypy: 34 errors, identical to main (all pre-existing, in uploads.py and older tests). Version in pyproject.toml untouched; notes for each item are under ## [Unreleased].

The server serialises loads per table and answers a second writer with
HTTP 409 RESOURCE_LOCKED + Retry-After. Add hotdata/loads.py with
DatabasesApi / ConnectionsApi subclasses that retry load_database_table
and load_managed_table on that code only, honouring Retry-After with
jitter under a LoadRetryPolicy budget (60s default), and raise
TableLockedError when it is exhausted. CONFLICT and
IDEMPOTENCY_KEY_CONFLICT are never retried; the code is matched, not the
message.

Export them as hotdata.DatabasesApi / ConnectionsApi via
patch_query_exports.py so a regen keeps the swap, and list loads.py in
.openapi-generator-ignore.
…ment

Configuration() now reads HOTDATA_API_KEY, HOTDATA_WORKSPACE and
HOTDATA_API_URL when the matching argument is None, the same variables
the CLI honours. Explicit arguments win; an empty variable is unset. The
change lives in the mustache template the generator uses for
configuration.py, so a regen keeps it.
A workspace-scoped operation sent without a workspace id came back as a
bare 404. ApiClient.update_params_for_auth now calls
hotdata._workspace.ensure_workspace, which raises MissingWorkspaceError
(an ApiValueError) naming workspace_id= and HOTDATA_WORKSPACE whenever
the operation's security lists WorkspaceId and no value would be sent.
Explicit X-Workspace-Id headers, _request_auth overrides and
BearerAuth-only endpoints (Workspaces) are untouched.

api_client.py is generated, so the hook is applied by
scripts/patch_workspace_guard.py from the regenerate workflow;
_workspace.py is generator-ignored and the error is exported from
hotdata via patch_query_exports.py.
…atus-only reply

wait_for_result polls with limit=0 and returns a response without rows,
which surprised callers. Say so prominently in its docstring and add
QueryApi.fetch_ready_result, which waits the same way and then fetches a
page of rows via get_result. Existing behaviour is unchanged.
Generated request models silently dropped unknown keyword arguments
(pydantic's default extra='ignore'), so a typo or a field from a newer
API never reached the server. scripts/patch_model_extra_forbid.py adds
extra='forbid' to every model used as a request body and to the models
nested in them, and the regenerate workflow re-applies it. Response
models are untouched so a newer server still cannot break an older SDK.

BREAKING CHANGE: constructing a request model with an unknown keyword
now raises pydantic.ValidationError.
@eddietejeda
eddietejeda requested a review from a team as a code owner October 15, 2026 19:38
@eddietejeda
eddietejeda requested review from zfarrell and removed request for a team October 15, 2026 19:38
@eddietejeda
eddietejeda enabled auto-merge (squash) October 15, 2026 19:39
@eddietejeda
eddietejeda merged commit 3373899 into main Oct 15, 2026
5 checks passed
@eddietejeda
eddietejeda deleted the feat/load-retry-and-env branch October 15, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant