feat(appkit): migrate analytics to the modular @databricks/sdk-* - #562
Draft
MarioCadenas wants to merge 1 commit into
Draft
feat(appkit): migrate analytics to the modular @databricks/sdk-*#562MarioCadenas wants to merge 1 commit into
MarioCadenas wants to merge 1 commit into
Conversation
Migrate the analytics stack (SQLWarehouseConnector + type-generator) off
the legacy monolithic @databricks/sdk-experimental onto the new modular
per-service @databricks/sdk-* SDK (v0.46.0, ESM-only), behind the existing
workspace-client facade seam. The two services analytics depends on —
warehouses and statementExecution — move together; every other service
still routes through the legacy client (mixed state by design).
- New packages/shared/src/workspace-client/modular.ts is the sole importer
of @databricks/sdk-* (oxlint no-restricted-imports boundary), mirroring
legacy.ts. Builds per-service WarehousesClient / StatementExecutionClient;
maps wrapper options -> ClientOptions (host scheme-normalization, PAT
empty-token guard, profile); stamps process-global client-info
(sanitized, best-effort).
- Connector + type-generator rewritten to the modular API: method renames
(getStatement -> getStatementResult, getStatementResultChunkN ->
getResultData), camelCase response model, CallOptions { signal }.
- statementExecution relies on a pinned pnpm patch that restores the
undocumented Reyden `attachment` field the SDK's unmarshal transform
would otherwise strip.
- Coerce the SDK's bigint row/byte counts back to number at the connector
boundary so INLINE + ARROW_STREAM results stay JSON-serializable
(cache / SSE frames).
- Read the modular ApiError's `.code` (not only the legacy `.errorCode`) so
the arrow disposition/format capability-rejection fallback still fires.
Verified against live warehouses (standard + Reyden serverless): JSON and
arrow (INLINE attachment + EXTERNAL_LINKS), OBO, warehouse auto-start, and
metric views. Full appkit + shared suite green (3877 tests).
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Migrates the analytics stack (
SQLWarehouseConnector+ type-generator) off the legacy monolithic@databricks/sdk-experimentalonto the new modular per-service@databricks/sdk-*SDK (v0.46.0, ESM-only), behind the existingpackages/shared/src/workspace-client/facade seam.Analytics doesn't touch the SDK directly — it delegates to
SQLWarehouseConnector, which uses two services:warehousesandstatementExecution. Both move together in this increment; every other service (files/genie/jobs/serving/currentUser/apiClient) still routes through the legacy client behind the same facade (the seam's designed incremental state).How
packages/shared/src/workspace-client/modular.ts— the sole importer of@databricks/sdk-*(oxlintno-restricted-importsboundary), mirroringlegacy.ts. Builds per-serviceWarehousesClient/StatementExecutionClient; maps wrapper options →ClientOptions(host scheme-normalization, PAT empty-token privilege-escalation guard, profile); stamps process-global client-info (sanitized, best-effort).getStatement→getStatementResult,getStatementResultChunkN→getResultData), camelCase response model,CallOptions { signal }for cancellation.patches/@databricks__sdk-statementexecution@0.46.0.patch) restores the undocumented Reydenattachmentresponse field that the SDK's generated unmarshal transform would otherwise strip. Version-pinned — re-apply on any SDK bump; an upstream fix is preferred long-term.Runtime bugs found + fixed (caught against live warehouses, not unit tests)
https://; a bareDATABRICKS_HOST→Invalid URL. Fixed vianormalizeHost.setProduct("@databricks/appkit")throws on@//; sanitized + wrapped best-effort.ApiError.code— the modular ApiError carries the Databricks error code on.code, not.errorCode; the connector now reads either, so the arrow disposition/format capability-rejection fallback (INLINE↔EXTERNAL_LINKS) fires again.bigint; Reyden's cached INLINE+ARROW_STREAM result then brokeJSON.stringify. Counts are coerced back tonumberat the connector boundary.Testing
appkit+sharedsuites green (3877 passed / 1 skipped), including new regression tests for the.coderead and bigint normalization.pnpm -r typecheck, lint, format clean..obo.sql), warehouse auto-start, metric views.Notes