fix(export): make generated Strands runtimes deployable - #2154
Conversation
|
Claude Security Review: no high-confidence findings. (run) |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## refactor #2154 +/- ##
============================================
- Coverage 97.14% 97.11% -0.03%
============================================
Files 519 519
Lines 35492 35509 +17
============================================
+ Hits 34478 34486 +8
- Misses 1014 1023 +9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
AgentCore Harness Review
Verdict: Looks good
Nice, tightly-scoped fix. I walked through the template updates, the exporter changes, and the new EC2-backed VPC lookup and didn't find anything that needs to change before merge. A few observations, none of which are blockers:
HarnessClient.resolveVpcIdFromSubnetsis wired cleanly throughAwsClients/CoreClientand the ec2 client is cached like the others; the new tests cover the multi-VPC / no-VPC / happy paths without mocking anything below the SDK boundary. 👍- The switch to a strict ARN regex in
parseHarnessArnis the right call, and the behavior change of always trusting the ARN's region (rather than falling back to CLI region) is now consistent betweenharnessIdFromArnandregionFromHarnessArn. The test inharness.test.ts(arn:aws:lambda:...) and the partition test inserviceHarness.test.ts(arn:aws-cn:...) cover the important edges. - The
main.pytemplate'slimits = { … } or Noneidiom is intentional: whenhasExecutionLimitsis true solely becausetimeoutSecondsis set, both{{#if}}branches inside the dict literal are stripped, so it evaluates toNone. As long asagent.stream_async(..., limits=None, ...)is accepted by strands-agents 1.54 (per the verification notes in the PR description, it is), this is fine. serviceHarness.tsnow surfaces$unknownunion members as export notes instead of silently dropping them (skills, memory, environment, environment artifact, filesystem configs). Good coverage inserviceHarness.test.ts.- Header credential names / python function names now hash their inputs, so
X-Api-KeyvsX_Api_Keyno longer collide — nice; that's exercised by the new "keeps normalized header names distinct" test. readStrandsVersionregex was updated to tolerate the newstrands-agents[extras]form.- No new features here that would need telemetry instrumentation; existing plumbing is untouched.
LGTM to merge.
ed3b929 to
bee4e65
Compare
|
Claude Security Review: no high-confidence findings. (run) |
| // results to. CloudWatch is a distinct service from the AgentCore data plane, | ||
| // so it gets its own client/factory rather than reusing `data`. | ||
| logs(config: ClientConfig): CloudWatchLogsClient; | ||
| ec2(config: ClientConfig): EC2Client; |
There was a problem hiding this comment.
Why do we need an EC2 client?
| .send(new GetHarnessCommand({ harnessId: id })); | ||
| } | ||
|
|
||
| async resolveVpcIdFromSubnets(subnetIds: string[], options: CoreOptions): Promise<string> { |
There was a problem hiding this comment.
Why do we need to do this? The API doesn't require this: https://docs.aws.amazon.com/bedrock-agentcore-control/latest/APIReference/API_NetworkConfiguration.html
| spec.networkConfig && | ||
| !spec.networkConfig.vpcId | ||
| ) { | ||
| spec.networkConfig.vpcId = await config.core.harness.resolveVpcIdFromSubnets( |
There was a problem hiding this comment.
I'm not sure this need to be in scope. The main thing is generating the code that the user can then use. The user can also configure VPC stuff on their own.
|
Claude Security Review: no high-confidence findings. (run) |
The pinned @aws/agentcore-cdk rejects additionalParams on every provider but lite_llm, and re-parses harness.json at synth. Dropping the CLI refinement moved that failure from `project add harness` to `project build`, where it surfaces as a raw zod dump — reachable via `project create --additional-params`, whose provider defaults to bedrock. Restore the refinement, and drop the field with an export note on the --arn path instead of hard-failing, since a harness authored outside this CLI can carry it.
Export layers the generated agent into the harness's image by writing a `FROM <containerUri>` Dockerfile, which turns a no-build harness into a CodeBuild build. CodeBuild's CreateProject needs an explicit vpcId and cannot infer one from subnets, so the exported project failed at `project build` with a raw zod dump from CDK synth. Neither source of a harness carries a vpcId — the service's VpcConfig has no such field, and a local containerUri harness is never built so its schema rightly does not demand one. Export is what creates the requirement, so add --vpc-id and fail before writing anything when it is needed and absent. No AWS lookup is involved: getHarness remains the only request on the --arn path.
The exported agent is a self-contained Strands application: its dependencies come from its own pyproject.toml and it defines its own entrypoint, so nothing in it comes from the source harness's image. Deriving a Container build from containerUri/dockerfile therefore bought a CodeBuild project, an ECR pull grant, and a VPC id the Runtime API cannot supply, for a base layer the agent never used. Export now always emits CodeZip and reports a source image or Dockerfile as an export note. Removes --vpc-id and --build from `project export harness`, along with resolveBuildType, resolveDockerfilePlan and buildDockerfileStub. Path-based skills are now rejected rather than noted: they name directories on the harness image's filesystem, which the exported agent does not have, so the files were simply absent at invocation. Republishing them from s3 or git is the supported path. Container-based export can return additively if a real need for it appears. ProjectRuntimeSchema's vpcId rule is kept — it guards `project add runtime` and hand-edited specs, which can still declare a Container build.
|
Claude Security Review: no high-confidence findings. (run) |
The optional provider params were rendered with inline conditionals whose leading indentation survived when every param was absent, so a plain `project create` scaffold — which sets none of them — emitted a stray closing paren at twelve spaces. Uses the standalone-conditional form the rest of the template already relies on.
|
Claude Security Review: no high-confidence findings. (run) |
Export rendered `templates/strands-http-python`, the same template `project create` scaffolds from, so every export fix landed in a file whose primary consumer is the regular Strands runtime. That coupling is why this PR was touching provider loading, MCP client construction and memory retrieval in a shared template to fix bugs only export could reach. Export now renders `templates/export-harness-python`, and `strands-http-python` is restored to its pre-aws#2146 state: the hooks/execution_limits.py that aws#2146 added is gone, and the Bedrock model line it extended with temperature/top_p is back to its original form. `project create` output is byte-identical to refactor, verified by rendering a scaffold on both commits and diffing. The export template also drops what export can never produce: the container Dockerfile and .dockerignore (export is always CodeZip) and the Anthropic provider block (export emits Bedrock, OpenAI, Gemini or LiteLLM).
The export template started as a copy of strands-http-python, so it carried branches for capabilities the export mapper hardcodes off: gateway, browser and code-interpreter tools (all reported as manual follow-ups rather than generated), payment and config-bundle (no harness concept), and path skills (now rejected outright). Removes those blocks and the render-context keys that fed them. Verified by generating exports for four harness shapes — default Bedrock, execution limits with sliding-window truncation, LiteLLM, and remote MCP with colliding header names plus s3/git skills — before and after: byte-identical output, so only unreachable branches were removed. The generated agent still resolves strands-agents 1.54 via uv sync and imports.
|
Claude Security Review: no high-confidence findings. (run) |
There was a problem hiding this comment.
Approving since some work is blocked on this, but I think we should come back to rethink this feature:
- a customer exporting their harness to runtime code would expect the same agent, but that doesn't seem to be how this works. We're generating a strands template, that may not match the underlying harness agent behavior, with the config we can extract from the harness. Additionally, the generated code will be highly complex due to the high degree of parametrization that I'm not sure it would be obvious to extend or work with.
- I'm wondering how much value this provides beyond mapping harness config to runtime config. If the mapping of configuration is the value, then perhaps we can simplify the implementation to reflect this by re-using the
addResourcefrom theadd runtimepath , rather than its own method on the manager that re-implements much of the same functionality. - some implementation details could be cleaned up, like unclear names, inconsistent conventions, and the
notespiece for accumulating warnings feels like an odd pattern.
| @@ -3,24 +3,23 @@ import z from "zod"; | |||
| import { InputValidationError } from "../../../errors/errors"; | |||
| import { HarnessSpecSchema, type HarnessSpec } from "../../../projectSchemas/harness"; | |||
There was a problem hiding this comment.
OOS, but I think we should move this testing into the handler tests, rather than on the internal implementation where possible. I think there's a fair bit of redundancy.
| export function mapHarnessToExportPlan(input: HarnessExportInput): HarnessExportPlan { | ||
| const { spec, targetAgentName, projectSpec } = input; | ||
| const notes: ExportNote[] = []; | ||
| const notes: ExportNote[] = [...(input.sourceNotes ?? [])]; |
There was a problem hiding this comment.
what exactly is a note? Based on the implementation, it looks like this could be described as a set of warnings?
| // application whose dependencies come from its own pyproject.toml, so it needs no image build | ||
| // and never reaches CodeBuild. A source image or Dockerfile is reported rather than rebuilt. | ||
| if (spec.containerUri || spec.dockerfile) { | ||
| const what = spec.containerUri |
There was a problem hiding this comment.
is there a better name for this variable? what feels ambiguous to me
| hasMemory: memory.provider !== undefined, | ||
| memoryEnvVarName: memory.provider?.envVarName, | ||
| memoryStrategies: memory.provider?.strategies ?? [], | ||
| memoryRetrievalTopK: |
There was a problem hiding this comment.
nit: could this be simplified to ?
memoryRetrievalTopK: memory.retrievalConfig?.topK?.toString(),
memoryRetrievalRelevanceScore: memory.retrievalConfig?.relevanceScore?.toString(),
| @@ -354,6 +334,7 @@ function resolveModel( | |||
| case "open_ai": | |||
| case "gemini": { | |||
| context.modelProvider = model.provider === "open_ai" ? "OpenAI" : "Gemini"; | |||
There was a problem hiding this comment.
for my own understanding, where do the 3 different formats of openai come from?
ex. open_ai, openai and OpenAI all appear here.
| flags: [ | ||
| flag("name", "the name of an in-project harness to export", z.string().optional()), | ||
| flag( | ||
| "arn", |
There was a problem hiding this comment.
does this mean I could do agentcore project export harness --arn and bring a harness from outisde the project into the project as a runtime?
maybe my mental model is off, but that feels backwards to export into a project.
| } | ||
|
|
||
| /** Extract the harness id from a validated harness ARN. */ | ||
| export function harnessIdFromArn(arn: string): string { |
There was a problem hiding this comment.
nit: these two functions feel like unnecessary indirection.
|
|
||
| // Progress goes to stderr, keeping stdout for machine output. Driven by | ||
| // hand because the result is the generator's return value. | ||
| const exportRun = config.projectManager.exportHarness(project, input); |
There was a problem hiding this comment.
I'm not convinced that export harness needs its own top level function in project manager.
My understanding of the feature is that we fetch the harness config information via getHarness, then scaffold a runtime with a "similar" config. Therefore, the addResource with runtime path should be able to support this since this runtime is just a special case of the general add runtime flow.
| @@ -1147,7 +1124,7 @@ function toProjectSpecKey(resourceType: ProjectResource) { | |||
| async function readStrandsVersion(agentDir: string): Promise<string> { | |||
There was a problem hiding this comment.
why do we need the strands version?
| @@ -5,26 +5,34 @@ import type { | |||
| import z from "zod"; | |||
There was a problem hiding this comment.
the naming of these files doesn't make their purpose clear. i.e. what lives in harness.ts vs serviceHarness.ts
Context
PR #2146 added
project export harness, which converts a deployed AgentCore Harness into an owned Strands runtime project, by local name or by service ARN. Bug bashing that merged behavior found eleven defects in the export path. Ten are real and fixed here; one is rejected as unreproducible.The headline problem was that exported agents did not run at all. Everything else was fidelity: settings the harness declared that the generated code silently discarded.
Review then established that export was also wrapping that code in a container build it never needed, which is where the CodeBuild and VPC machinery in earlier revisions of this PR came from. That is all removed, and export now renders its own template instead of the one
project createscaffolds from.Based on
refactor, notmain.mainis not the bug baseline: its template floatsstrands-agents >= 1.15.0, which currently resolves 1.54 and masks bugs 1 and 2. It still ships the rest.Bugs fixed
1. Generated agents called Strands APIs that did not exist in the pinned version
pyproject.tomlpinnedstrands-agents ~= 1.15.0whilemain.pyused APIs from much later releases. A harness with skills producedImportError: cannot import name 'AgentSkills'onimport main. A harness with sliding-window truncation raisedTypeError: unexpected keyword argument 'per_turn'on every invocation. Neither failed at export or build; both failed at runtime.The template now pins
strands-agents ~= 1.54.0. That is the floor, not a jump to latest:AgentSkills,per_turnand cancellation exist by 1.30 and native per-invocationlimitsby 1.45, butstream_async(..., cancel_signal=...)is absent through 1.53 and first appears in 1.54.2. OpenAI and Bedrock Mantle exports could not install their dependencies
The per-provider dependency list pinned
openai ~= 1.0.0, which cannot satisfy themcp/anyiograph thatstrands-agentsrequires.uv syncfailed outright, so those exports never reached a runnable state.Provider dependencies now come from the Strands extras (
[openai],[gemini],[litellm]) instead of a hand-maintained list, andmcpis bounded as>= 1.23.0, < 2.0.0, mirroring the constraintstrands-agents 1.54.0itself declares.3. Execution limits leaked across invocations and the timeout could not stop the agent
The generated code carried a custom hook that accumulated turn and token counts on a cached agent, so the second invocation in a session started with the first one's usage already spent and failed. The timeout path called
agent.cancel(), which does not exist on the pinnedAgent.The hook is gone. The generated agent passes native per-invocation
limitsand a request-scopedcancel_signal, so each invocation gets fresh limits and a timeout that actually interrupts a waiting stream.4. Remote MCP identity was resolved at module import
Header credentials were fetched while building the client, which
main.pydoes at import.import maintherefore calledCreateWorkloadIdentitywith no request context and crashed.The credential lookup moved inside the MCP transport factory, so it runs when the transport opens.
import mainnow performs no identity call.5. Provider settings were dropped, and OpenAI used the wrong adapter
temperature,topP,topKandmaxTokenswere rendered for Bedrock only; OpenAI, Gemini and LiteLLM silently ignored them. A harness withapiFormat: responseswas also instantiated through the chat-completions adapter.All four providers now render the requested parameters, and
apiFormat: responsesinstantiatesOpenAIResponsesModel.6. MCP header names that normalize identically collapsed onto one credential
Credential, env-var and function names were derived by snake-casing the header.
X-Api-KeyandX_Api_Keyboth becamex_api_key, so two distinct headers shared one credential and the agent could send the wrong secret.Names are now derived from a stable hash of the header, producing distinct identifiers for headers that differ only in separator.
7. Memory retrieval tuning was replaced by hardcoded defaults
A harness could request
topKandrelevanceScore; the generatedRetrievalConfighardcodedtop_k=3, relevance_score=0.5regardless, with no note that the request had been ignored.messagesCounthas no Strands equivalent and was also dropped silently.Requested values now render, and
messagesCountis recorded inEXPORT_NOTES.mdrather than discarded.8. Unrecognized service fields were dropped while the notes claimed nothing was left to do
The service models several fields as unions with an
$unknownvariant. The mapper filtered every variant it did not recognize — includingsystemPromptblocks — and emitted nothing, soEXPORT_NOTES.mdcould still report "No manual steps required" after losing content.Unknown members are now reported in the export notes.
9.
--arnaccepted ARNs from other servicesValidation checked only that the string ended in
:harness/<id>, so a wrong-service or malformed ARN reached the service fetch and failed there.The service, partition, account and resource shape are all validated before the fetch.
10. Rejected: service-ARN VPC/container export drops the required VPC ID
Not reproduced. Baseline and head both preserve the service's subnets and security groups, and the Runtime API's
NetworkConfigurationhas novpcIdfield to drop. An EC2DescribeSubnetslookup written for this claim was removed inb3fbb3dd, and the container build that later raised the same VPC question is gone as well — see below.Structural changes from review
Export always emits a CodeZip runtime
Export derived a Container build whenever the source harness had a
containerUrior adockerfile, generated aFROM <containerUri>Dockerfile, and layered the agent on top. Nothing in the generated agent uses that base image: it declares its own dependencies inpyproject.tomland defines its own entrypoint. The coupling only bought a CodeBuild project, an ECR pull grant the old CLI documented as a manual step, and a VPC id the Runtime API cannot supply.Export now always emits
CodeZipwithruntimeVersion: PYTHON_3_14. A source image or Dockerfile is reported inEXPORT_NOTES.mdinstead of rebuilt.--vpc-idand--buildare removed, along withresolveBuildType,resolveDockerfilePlanandbuildDockerfileStub.Path-based skills are rejected as a consequence: they name directories on the harness image's filesystem, which the exported agent does not have, so those files were simply absent at invocation. Export now fails with a message pointing at the s3 and git skill variants rather than shipping an agent that breaks on first use.
Export has its own template
Export rendered
templates/strands-http-python, the same templateproject createscaffolds from, so every export fix landed in a file whose primary consumer is the regular Strands runtime.Export now renders
templates/export-harness-python, andstrands-http-pythonis restored to its pre-#2146 state — thehooks/execution_limits.pythat #2146 added is removed, and the Bedrock model line it extended is back to its original form. The export template also drops what export can never produce: the container Dockerfile and.dockerignore, the Anthropic provider block, and the gateway, browser, code-interpreter, payment, config-bundle and path-skill branches, together with the render-context keys that fed them.ProjectRuntimeSchemarequires a vpcId for container builds in VPC modeProjectRuntimeSchemaalready mirrors the pinned CDK's Container+VPC security-group cap but not thevpcIdrequirement sitting beside it in the same CDKsuperRefine. Export no longer produces Container builds, butproject add runtime --build Container --network-mode VPC --network-config '{subnets, securityGroups}'and hand-edited specs still can, and they were writing projects that fail at synth. Fixed at the schema, which covers every writer ofagentcore.json.Validation
Automated
bun install --frozen-lockfilebun test src— 2,749 passed, 0 failed across 197 filesbun run typecheckbun run lint:checkbun run format:checkbun run buildThe regular template is untouched
A
project createPython/Strands scaffold was generated fromrefactorand from this branch and diffed, with and without memory. Output is byte-identical, so nothing in this PR changes whatproject createproduces.Removing unreachable template branches changed no output
Exports were generated for four harness shapes before and after pruning the export template — default Bedrock; execution limits with sliding-window truncation; LiteLLM with an API base; and remote MCP with colliding
X-Api-Key/X_Api_Keyheader names plus s3 and git skills. All four are byte-identical, confirming only unreachable branches were removed. The check was repeated after the matching render-context keys were deleted.Generated Python
Real exports were generated, then
uv sync,compileallandimport mainrun for: Bedrock with skills, sliding-window truncation, execution limits, timeout and credential-backed remote MCP; OpenAI Responses; Gemini; LiteLLM; and Bedrock Mantle Responses. All five resolvedstrands-agents 1.54.0and imported under Python 3.14.3 —openai 2.54.0,google-genai 2.20.0,litellm 1.96.0. A generated-module probe invoked the Bedrock entrypoint twice in one session and confirmed fresh cancellation signals with identical per-invocation limits. On the baseline, the OpenAI and Mantle environments fail to resolve and the sliding-window, skills and MCP probes fail with the exact behavior described above.A final end-to-end check against the new export template:
project export harnesssucceeded,uv syncresolvedstrands-agents ~= 1.54.0, andimport mainsucceeded.Live AWS
Account
603141041947,us-east-1:ExportHarnessFix0831_source-jBo3nhtmm2to READY and invoked itus-west-2, confirming ARN-region precedenceExportHarnessFix0831_exported_agent-hLU1ha80fdandExportHarnessFix0831_exported_arn_agent-ItzBMG47U0, then invoked each twice in the same session; all four returned HTTP 200,end_turnand the requested markerExportHarnessFix0831_mcp_runtime-6ZSyRO2yME, confirmed module import succeeds with no request identity context, then invoked it successfullyAgentCore-ExportHarnessFix0831-defaultdeleted, no matching runtimes or harnesses remain, disposable credential provider deletedSchema check against the pinned CDK
A project with a VPC container runtime was parsed with
AgentEnvSpecSchemafrom@aws/agentcore-cdk@0.1.0-alpha.45, the version generated projects pin. A runtime with novpcIdis rejected withnetworkConfig.vpcId is required for Container builds in VPC mode; one with avpcIdis accepted.Out of scope
--buildremoved there is no way to request one. Nothing in a generated agent needs an image, and the option carried real cost.main.mainmasks bugs 1 and 2 through its floating pin but still ships the rest, and its export mapper derives Container builds the same way. Tracked separately.uv syncandimport mainresults above come from an out-of-repo audit harness.refactordeploy path treatsagentcore.jsoncredentials as references. The export note now states the provider must already exist rather than implying deploy creates it.