test(playground): smoke-test the server plugins and showcase the testing kit - #542
Draft
IamGalymzhan wants to merge 2 commits into
Draft
test(playground): smoke-test the server plugins and showcase the testing kit#542IamGalymzhan wants to merge 2 commits into
IamGalymzhan wants to merge 2 commits into
Conversation
IamGalymzhan
marked this pull request as draft
August 20, 2026 10:06
IamGalymzhan
force-pushed
the
feat/dev-playground-smoke-tests
branch
from
August 27, 2026 12:25
f909a49 to
868331a
Compare
IamGalymzhan
changed the base branch from
feat/testing-kit-harness
to
feat/testing-kit-helpers
August 27, 2026 12:25
This was referenced Aug 27, 2026
IamGalymzhan
marked this pull request as ready for review
August 27, 2026 12:29
IamGalymzhan
force-pushed
the
feat/dev-playground-smoke-tests
branch
2 times, most recently
from
August 28, 2026 11:35
5919db4 to
705c30f
Compare
The playground's server plugins had no tests at all. `tests/` holds Playwright specs that intercept `/api` at the browser boundary (`page.route` + `fulfill`), so the Express server never runs in them — route handlers, `execute()` interceptors, and `executeStream` are all unexercised. These five tests cover that side using the harness from #540: real HTTP, faked data plane, no workspace, no credentials, no network. Adds a `dev-playground` vitest project so `pnpm test` (and CI's Unit Tests job) picks them up: 4517 -> 4522. The `tests/**` exclusion is load-bearing — without it vitest's default `**/*.spec.ts` glob collects the Playwright specs and they fail on import with "Playwright Test did not expect test.describe() to be called here". Two things worth knowing, both found by writing this: - `telemetry-example-plugin` really calls `fetch("https://example.com")` in its external-api span, so a naive test needs the internet. The suite stubs non-loopback fetches and passes loopback through, since that is how the harness reaches its own server. Verified hermetic under a socket guard that throws on any non-loopback connect. - `expectStream` buffers a source to completion and throws on timeout rather than returning partial events, so it cannot assert on a long-lived stream. The reconnect stream is five messages three seconds apart, so the SSE test reads one payload and hangs up instead of costing ~12s. Scoped to smoke coverage deliberately. Not included: reconnection replay (`executeStream`'s ring buffer and `Last-Event-ID` handling is the real prize and deserves its own tests), and a `typecheck` script for the app — the existing tsconfig reports 2128 errors from the client's unset `--jsx`, and a server-scoped one still has 13 pre-existing errors that CI has never seen because `pnpm -r typecheck` skips this workspace. Signed-off-by: Galymzhan <zhangazy2004@gmail.com>
IamGalymzhan
force-pushed
the
feat/dev-playground-smoke-tests
branch
from
August 31, 2026 08:20
705c30f to
7df788e
Compare
…ugins smoke.test.ts proves the playground's plugins boot and answer; this adds a worked example of each recommended `@databricks/appkit/testing` helper, so the reference app dogfoods the kit the way the docs tell customers to test their own plugins. One describe per helper: getMock, createMockWorkspaceClient, createApiError, useTestApp, withEnv, useTestCache/resetTestCache, createTestPlugin, createTestPluginContext, expectStream, and useServiceContextMock. Driven against real plugins where the seam is clean: files.getMetadata for getMock and the createApiError error path, reconnect for useTestApp, genie for createTestPlugin. Where no playground plugin fits hermetically the demo mirrors the docs — expectStream on a bounded generator (the reconnect stream is long-lived, so smoke.test.ts reads it by hand), and cross-plugin tool dispatch straight through the context (agents needs a live model). Signed-off-by: Galymzhan <zhangazy2004@gmail.com>
IamGalymzhan
force-pushed
the
feat/dev-playground-smoke-tests
branch
from
August 31, 2026 08:26
7df788e to
5f35ad8
Compare
IamGalymzhan
marked this pull request as draft
August 31, 2026 09:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
Each PR targets the one above it, so the diff shown here is only the delta on top of #555. Rebased onto the current #555 tip, so the helpers it adds (
useTestApp, the kit-faked workspace client) are available here.Summary
The playground's server plugins had no server-side tests.
apps/dev-playground/tests/holds Playwright specs that intercept/apiat the browser boundary (page.route+route.fulfill), so the Express server never runs in them — route handlers,execute()interceptors, andexecuteStreamare unexercised.Two suites cover that side with the harness from #540 — real HTTP, faked data plane, no workspace, no credentials, no network — and dogfood the kit the way the docs tell customers to test their own plugins.
smoke.test.ts— the plugins boot and answerGET /api/reconnecttext/event-streamand emits its first messagePOST /api/telemetry-examples/combinedTelemetryProviderand threadsuserIdthroughlakebase-examplesunconfiguredPGHOST/LAKEBASE_ENDPOINT, and the app still bootstesting-kit.integration.test.ts— a worked example of each recommended helperOne
describeper helper, driven against a real plugin where the seam is clean:getMock(+responses)metadataroute →files.getMetadata, asserted onapp.clientcreateMockWorkspaceClientundefinedcreateApiErrorApiError, and seeded as a route failureuseTestAppwithEnvuseTestCache/resetTestCacheCacheManager— stable & per-user keys, hit-then-misscreateTestPlugingeniefactory → a configured instancecreateTestPluginContext(+createMockRequest)asUser, token-less rejectexpectStream(+createMockResponse)toEmit/toEmitExactlyon a bounded sourceuseServiceContextMockWhere no playground plugin fits hermetically, the demo mirrors the docs:
expectStreamon a bounded generator (reconnect's stream is long-lived, sosmoke.test.tsreads it by hand), and tool dispatch straight through the context (agents needs a live model).Adds a
dev-playgroundvitest project sopnpm testand CI's Unit Tests job pick both suites up. Thetests/**exclusion is load-bearing — without it vitest's default glob collects the Playwright specs, which fail on import.This pull request and its description were written by Isaac.