Repository navigation
feat: retry locked loads, read env vars, clear workspace error - #219
Merged
Merged
Conversation
The server serialises loads per table and answers a second writer with HTTP 409 RESOURCE_LOCKED + Retry-After. Add hotdata/loads.py with DatabasesApi / ConnectionsApi subclasses that retry load_database_table and load_managed_table on that code only, honouring Retry-After with jitter under a LoadRetryPolicy budget (60s default), and raise TableLockedError when it is exhausted. CONFLICT and IDEMPOTENCY_KEY_CONFLICT are never retried; the code is matched, not the message. Export them as hotdata.DatabasesApi / ConnectionsApi via patch_query_exports.py so a regen keeps the swap, and list loads.py in .openapi-generator-ignore.
…ment Configuration() now reads HOTDATA_API_KEY, HOTDATA_WORKSPACE and HOTDATA_API_URL when the matching argument is None, the same variables the CLI honours. Explicit arguments win; an empty variable is unset. The change lives in the mustache template the generator uses for configuration.py, so a regen keeps it.
A workspace-scoped operation sent without a workspace id came back as a bare 404. ApiClient.update_params_for_auth now calls hotdata._workspace.ensure_workspace, which raises MissingWorkspaceError (an ApiValueError) naming workspace_id= and HOTDATA_WORKSPACE whenever the operation's security lists WorkspaceId and no value would be sent. Explicit X-Workspace-Id headers, _request_auth overrides and BearerAuth-only endpoints (Workspaces) are untouched. api_client.py is generated, so the hook is applied by scripts/patch_workspace_guard.py from the regenerate workflow; _workspace.py is generator-ignored and the error is exported from hotdata via patch_query_exports.py.
…atus-only reply wait_for_result polls with limit=0 and returns a response without rows, which surprised callers. Say so prominently in its docstring and add QueryApi.fetch_ready_result, which waits the same way and then fetches a page of rows via get_result. Existing behaviour is unchanged.
Generated request models silently dropped unknown keyword arguments (pydantic's default extra='ignore'), so a typo or a field from a newer API never reached the server. scripts/patch_model_extra_forbid.py adds extra='forbid' to every model used as a request body and to the models nested in them, and the regenerate workflow re-applies it. Response models are untouched so a newer server still cannot break an older SDK. BREAKING CHANGE: constructing a request model with an unknown keyword now raises pydantic.ValidationError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the SDK gaps reported by Tenki. Five commits, one per item; every hand-written piece is generator-ignored or re-applied by a patch script from
regenerate.yml, so a regen keeps it.1. Retry locked loads (
feat(loads))The server serialises loads per table; a second writer gets
409 {"error":{"code":"RESOURCE_LOCKED",...}}+Retry-After. Newhotdata/loads.pysubclasses the generatedDatabasesApi(load_database_table) andConnectionsApi(load_managed_table, the only other/loadsoperation) and retries on 409 + codeRESOURCE_LOCKEDonly, matched on the code (fromexc.dataor the raw JSON body), never the message.Retry-Afteris honoured (uncapped, jitter on top), else exponential backoff;LoadRetryPolicydefaults to a 60 s budget / 10 retries; exhaustion raisesTableLockedError(a 409ApiExceptionwithattemptsand the last response).CONFLICT/IDEMPOTENCY_KEY_CONFLICT/ other errors pass through untouched. Exported ashotdata.DatabasesApi/ConnectionsApiviapatch_query_exports.py, same swap asQueryApi.2. Environment variables (
feat(config))Configuration()defaultsapi_keyfromHOTDATA_API_KEY,workspace_idfromHOTDATA_WORKSPACEandhostfromHOTDATA_API_URLwhen the argument isNone, matching the CLI. Explicit args win; an empty variable is unset.configuration.pyis generated from the in-repoconfiguration.mustache, so the change is in the template too (a test pins that).3. Clear no-workspace error (
feat(client))ApiClient.update_params_for_authnow callshotdata._workspace.ensure_workspace: if the operation's security listsWorkspaceIdand no value would be sent, it raiseshotdata.MissingWorkspaceError(anApiValueError) before the request goes out, namingworkspace_id=,HOTDATA_WORKSPACEand theX-Workspace-Idheader. Endpoints withBearerAuthonly (the Workspaces endpoints, per the spec) are unaffected, as are calls passing the header via_headersor overriding auth via_request_auth. The spec has no session scheme any more, so there is nothing else to accept. Applied by newscripts/patch_workspace_guard.py(wired into the workflow).4.
wait_for_resultdocs +fetch_ready_result(feat(query))wait_for_result's docstring now states up front that it polls withlimit=0and returns a status-only response with no rows. NewQueryApi.fetch_ready_result(result_id, x_database_id, offset=, limit=, format=)does the same wait and then oneget_resultfor the rows. Existing behaviour unchanged.5. Reject unknown model fields (
feat(models)!, breaking)scripts/patch_model_extra_forbid.pyaddsextra="forbid"to every model used as a request body plus the models nested in them (29 models; theColumnDefinitiononeOf wrapper has no plain fields and is skipped). Unknown kwargs now raisepydantic.ValidationError. Response models are deliberately left alone so a newer server never breaks an older SDK; the existing forward-compat test still passes. The whole suite, the SDK's own code and the integration tests (checked statically) pass no unknown fields. Changelog entry is marked Breaking under[Unreleased]### Changed.Tests
tests/test_loads.py(16): retry/Retry-After/backoff/exhaustion, code-not-message matching, other 409s and non-409s untouched, arg forwarding, export wiring.tests/test_config_env.py(7): env defaults, precedence, empty-is-unset, wire check, template parity.tests/test_workspace_guard.py(10): fails before sending, env/arg/header/_request_authpaths, Workspaces endpoints still work, workflow wiring.tests/test_query.py(+4): status-only contract,fetch_ready_resultwait-then-fetch, paging args, failure propagation.tests/test_model_extra_forbid.py(6): unknown kwarg raises, aliases still work, responses still ignore extras, closure coverage, idempotency + workflow wiring.pytest test tests --ignore=tests/integration: 422 passed.mypy: 34 errors, identical tomain(all pre-existing, inuploads.pyand older tests). Version inpyproject.tomluntouched; notes for each item are under## [Unreleased].