Repository navigation
Reorganize js sdk wrappers #1011
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| # Braintrust SDK Wrappers | ||
|
|
||
| This directory contains wrappers for various AI frameworks and SDKs. All wrappers are **exported from the main `braintrust` package** and ship as part of the regular npm distribution. | ||
|
|
||
| ## Directory Structure | ||
|
|
||
| ### Standard Wrappers | ||
|
|
||
| Most wrappers are single files that live directly in this directory: | ||
|
|
||
| - `anthropic.ts` - Anthropic SDK wrapper | ||
| - `oai.ts` - OpenAI SDK wrapper | ||
| - `google-genai.ts` - Google GenAI wrapper | ||
| - etc. | ||
|
|
||
| ### Isolated Test Wrappers | ||
|
|
||
| Some wrappers have their own subdirectories with `package.json` files: | ||
|
|
||
| - `ai-sdk-4/` - AI SDK v4.x wrapper | ||
| - `ai-sdk-5/` - AI SDK v5.x wrapper (middleware + utilities) | ||
| - `mastra/` - Mastra agent wrapper | ||
| - `claude-agent-sdk/` - Claude Agent SDK wrapper | ||
|
|
||
| **Important**: These subdirectories contain **private test-only packages** (`"private": true`). The code is still exported from the main `braintrust` package - the separate `package.json` files exist solely to run tests with specific dependency versions. | ||
|
|
||
| ## Why Separate package.json Files? | ||
|
|
||
| Some integrations need to be tested against specific versions of their underlying SDKs: | ||
|
|
||
| - **AI SDK v4 vs v5**: Breaking changes between major versions require separate test environments | ||
| - **Mastra**: Uses AI SDK v5 and has specific version requirements | ||
| - **Claude Agent SDK**: Has its own versioning independent of Anthropic's main SDK | ||
|
|
||
| ### What Gets Published? | ||
|
|
||
| **Everything in this directory ships in the main `braintrust` npm package.** The subdirectory `package.json` files are marked `"private": true` and are **never published to npm**. | ||
|
|
||
| Users install just one package: | ||
|
|
||
| ```bash | ||
| npm install braintrust | ||
| ``` | ||
|
|
||
| And can import any wrapper: | ||
|
|
||
| ```typescript | ||
| import { wrapAISDK, wrapMastraAgent, wrapClaudeAgentSDK } from "braintrust"; | ||
| ``` | ||
|
|
||
| ### Running Tests | ||
|
|
||
| **Main package tests:** | ||
|
|
||
| ```bash | ||
| cd js | ||
| pnpm test # Excludes subdirectory tests | ||
| ``` | ||
|
|
||
| **Subdirectory tests** (with isolated dependencies): | ||
|
|
||
| ```bash | ||
| cd js/src/wrappers/ai-sdk-4 && pnpm test | ||
| cd js/src/wrappers/ai-sdk-5 && pnpm test | ||
| cd js/src/wrappers/mastra && pnpm test | ||
| cd js/src/wrappers/claude-agent-sdk && pnpm test | ||
| ``` | ||
|
|
||
| ## Shared Utilities | ||
|
|
||
| ### `ai-sdk-shared/` | ||
|
|
||
| Contains shared utility functions used by AI SDK wrappers and other integrations: | ||
|
|
||
| - `utils.ts` - Common utilities for AI SDK integrations (token counting, provider detection, etc.) | ||
| - `utils.test.ts` - Tests for shared utilities | ||
|
|
||
| These utilities are imported by: | ||
|
|
||
| - `ai-sdk-4/` and `ai-sdk-5/` wrappers | ||
| - `mastra/` wrapper (uses AI SDK v5) | ||
| - `claude-agent-sdk/` wrapper (for token utilities) | ||
|
|
||
| ## Adding a New Wrapper | ||
|
|
||
| ### For Standard Wrappers (Single File) | ||
|
|
||
| 1. Create `your-wrapper.ts` in this directory | ||
| 2. Create `your-wrapper.test.ts` for tests | ||
| 3. Export from `src/exports-node.ts`: | ||
| ```typescript | ||
| export { wrapYourSDK } from "./wrappers/your-wrapper"; | ||
| ``` | ||
| 4. Add test exclusion to main `package.json` if needed | ||
|
|
||
| ### For Isolated Test Wrappers (Separate package.json) | ||
|
|
||
| Only needed if you require specific SDK versions for testing: | ||
|
|
||
| 1. Create subdirectory: `js/src/wrappers/your-wrapper/` | ||
| 2. Move files: `your-wrapper.ts`, `your-wrapper.test.ts` | ||
| 3. Create `package.json` (marked `"private": true`) with specific dependencies | ||
| 4. Create `vitest.config.js` and `tsconfig.json` (copy from `ai-sdk-4/`) | ||
| 5. Fix imports to use relative paths (`../../logger`, `../ai-sdk-shared/utils`, etc.) | ||
| 6. Update `src/exports-node.ts`: | ||
| ```typescript | ||
| export { wrapYourSDK } from "./wrappers/your-wrapper/your-wrapper"; | ||
| ``` | ||
| 7. Run `pnpm install` in the subdirectory | ||
|
|
||
| ## GitHub Actions | ||
|
|
||
| All wrapper subdirectories with their own `package.json` are tested automatically in CI via a single workflow file: `.github/workflows/reusable-test-sdk-wrappers.yaml` | ||
|
|
||
| ### How It Works | ||
|
|
||
| The workflow uses a **matrix strategy** to test all wrappers in parallel: | ||
|
|
||
| ```yaml | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| wrapper: | ||
| - ai-sdk-4 | ||
| - ai-sdk-5 | ||
| - mastra | ||
| - claude-agent-sdk | ||
| ``` | ||
|
|
||
| Each wrapper gets its own job that: | ||
|
|
||
| 1. Checks out the repository | ||
| 2. Sets up Node.js and pnpm | ||
| 3. Installs dependencies in the wrapper's subdirectory | ||
| 4. Runs `pnpm test` with the isolated dependencies | ||
|
|
||
| ### Adding a New Wrapper to CI | ||
|
|
||
| When adding a new isolated test wrapper: | ||
|
|
||
| 1. Follow the steps in "Adding a New Wrapper" above | ||
| 2. Add the wrapper directory name to the matrix in `.github/workflows/reusable-test-sdk-wrappers.yaml`: | ||
| ```yaml | ||
| matrix: | ||
| wrapper: | ||
| - ai-sdk-4 | ||
| - ai-sdk-5 | ||
| - mastra | ||
| - claude-agent-sdk | ||
| - your-new-wrapper # Add here | ||
| ``` | ||
|
|
||
| The workflow is integrated into the main CI pipeline (`.github/workflows/braintrust-ci.yaml`) and runs automatically on pull requests. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "name": "@braintrust/ai-sdk-4-tests", | ||
| "version": "0.0.1", | ||
| "private": true, | ||
| "description": "Test infrastructure for AI SDK v4.x compatibility", | ||
| "scripts": { | ||
| "test": "vitest run" | ||
| }, | ||
| "dependencies": { | ||
| "@ai-sdk/provider": "^1.1.3" | ||
| }, | ||
| "devDependencies": { | ||
| "ai": "4.3.19", | ||
| "@types/node": "^20.10.5", | ||
| "typescript": "5.4.4", | ||
| "vitest": "^2.1.9", | ||
| "vite-tsconfig-paths": "^4.3.2" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| { | ||
| "extends": "../../../tsconfig.json", | ||
| "compilerOptions": { | ||
| "baseUrl": ".", | ||
| "rootDir": "../../..", | ||
| "outDir": "./dist" | ||
| }, | ||
| "include": [ | ||
| "./**/*.ts", | ||
| "../../../src/**/*.ts" | ||
| ], | ||
| "exclude": [ | ||
| "node_modules/**", | ||
| "dist/**" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| import tsconfigPaths from "vite-tsconfig-paths"; | ||
|
|
||
| const config = { | ||
| plugins: [tsconfigPaths()], | ||
| resolve: { | ||
| extensions: [".ts", ".tsx", ".mts", ".js", ".mjs", ".jsx", ".json"], | ||
| }, | ||
| }; | ||
| export default config; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i wonder if we could just have the main package.json delegate to the underlying ones.
i.e.
tl;dr i wonder if there's a nice way to get this without each of these make targets (always seemed odd)
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, i'd be fine with just moving all of this in
package.json. I'll make a ticket