Skip to content

feat(platform): fix init targeting and make SSE streams resumable - #5

Merged
piotrlaczkowski merged 9 commits into
mainfrom
claude/agentomatic-cli-templates-stb46v
Sep 9, 2026
Merged

piotrlaczkowski merged 9 commits into
mainfrom
claude/agentomatic-cli-templates-stb46v

Conversation

@piotrlaczkowski

@piotrlaczkowski piotrlaczkowski commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Three related pieces: fixing where agentomatic init puts code, making the platform's SSE streams survive a dropped connection, and hardening that new surface for production.


Part 1 — CLI scaffolding

The templates were never broken — all 16 generate real, working code. The bug was where it landed.

agentomatic init resolved agents/ against the current working directory. Run it from a subdirectory of a project — or from the parent it had just been created in — and it silently built a second, parallel project, leaving the real agents/ empty while reporting success:

$ agentomatic new my_platform
$ agentomatic init hello --template basic     # forgot to cd
   📍 Location: agents/hello                   # ← a NEW agents/ beside the project
$ ls my_platform/agents/
.gitkeep  __init__.py                          # ← empty

--dir made it worse: the bootstrap wrote stacks/, main.py and .agentomatic-stack into whatever directory happened to contain the target — a home directory if the flag pointed at one.

Target resolution

  • Resolve the enclosing project by walking up from the current directory, stopping at a repository root or $HOME. Markers are specific so a scaffold cannot escape into a lookalike ancestor.
  • Use a lone project found in a subdirectory (the new → forgot-to-cd case) and print which root was chosen; name the candidates and stay put when several match.
  • Add --here to opt out; confine --dir to placing the package.
  • Fix the same cwd assumption in add connection and add ingestion.

Making a scaffolded project run and deploy as advertised

  • Prompt for NAME instead of hard-exiting on a bare agentomatic init; reject names that are not importable package names.
  • Report the absolute location and verify files reached disk.
  • Bootstrap the platform main.py — deploy renders uvicorn main:app, so a project without one built a container that could not boot. An existing main.py is never overwritten.
  • Detect the model a local OpenAI-compatible server has loaded and bake it into stacks/local.yaml. The hardcoded Qwen3.5-9B-MLX-4bit meant any other loaded model answered the first invoke with HTTP 400.
  • Add openai to the all extra — the default local stack's omlx provider needs it, so agentomatic[all] scaffolded a project whose every LLM call failed.

Part 2 — Resumable streams

TaskManager._emit returned early when a task had no subscribers, so events emitted while a client was disconnected were never created, let alone retained. A dropped SSE connection could only be recovered as far as the task's current state.

  • TaskEvent gains a monotonic per-task sequence, gapless from 1, published as the SSE id:. A browser EventSource resumes through Last-Event-ID with no client code; ?since= does the same for other clients.
  • New TaskEventLog: an ordered, bounded replay buffer with an in-memory default, a null implementation, and a provider registry so a shared backend can be registered for cross-worker replay (the interface ships, not a vendor client).
  • A truncated resume point is announced rather than served as if complete.

This lands for every target type at once — agents, pipelines, plugins, endpoints and ingestion share one task manager, so a pipeline's per-step progress resumes exactly like an agent's.

A2A gains GET /a2a/tasks/{id}/events with the same contract, and /.well-known/agent.json now renders the same card as /{agent}/card through one builder — the discovery document previously omitted capabilities entirely and could not advertise streaming at all.

Other SSE surfaces: new agentomatic.streaming provides one encoder, a stream identity and a bounded replay buffer. /invoke/stream numbers and records its frames and returns X-Stream-Id; GET /invoke/stream/{stream_id} re-sends them. Studio's streams are numbered through the same helper.

What this deliberately does not claim

/invoke/stream replay returns what was produced. It does not restart the run: the agent execution is bound to the request Starlette cancels on disconnect. The docs say so and point at the durable path (submit as a task, follow its events) rather than implying a guarantee the architecture cannot keep.


Part 3 — Production hardening

A security review of this branch found — and reproduced — a cross-agent authorization bypass in the replay route added in Part 2. GET /{agent}/invoke/stream/{stream_id} resolved the id against a single process-wide buffer with no check that the stream belonged to the agent in the path. Zero-trust derives its policy purely from that agent segment, so a caller permitted on a permissive agent could name a restricted agent's stream id on their own route and receive the restricted agent's frames. Studio run frames shared the namespace, and Studio run ids are enumerable through a path zero-trust does not cover.

Fixed by restoring the pattern the codebase already used two files over (run.agent_name != name → 404):

  • Frames carry an ownership tag pinning them to the producing agent and the principal they were produced for.
  • The replay route answers 404 for a stream it may not read — the same answer as a nonexistent id, so it never confirms someone else's stream exists.
  • Studio frames are tagged separately, so a run id is not valid on an agent route.
  • create_streaming_response takes an owner and now requires a fresh new_stream_id.

Three more production defects, all found by review rather than by tests failing:

  • Retention was not a memory bound. Both buffers capped entries by count only — ~300 MB of ordinary traffic, more with long answers. Byte budgets added at both per-stream and total level; the deques lost maxlen because an implicit drop evicted an entry without its size leaving the running total.
  • Studio keyed its resume stream on thread_id — a client-supplied path parameter, so a second resume continued the first's numbering and a caller could name another thread's frames.
  • The A2A stream read the task record once per replayed event, though it is only needed for the terminal frame — N queries with the SQL store.

Plus: retention is sized from the environment like every other platform switch (AGENTOMATIC_TASK_EVENT_*, AGENTOMATIC_STREAM_*; a bad value logs and falls back), and /status reports what the buffers hold so an operator can size them.


Testing

  • All 16 templates: scaffold → compile → register with the platform.
  • Full journey on a fresh pip install -e ".[all]": scaffold, boot, agent registers, discovery advertises streaming, live stream numbered, replay returns exactly the gap, task stream resumes from Last-Event-ID, /status reports occupancy.
  • The reported exploit reproduced and asserted closed, including the indistinguishable 404.
  • Memory bounds asserted under pressure; auth asserted on every new route.
  • ~190 new tests across five files.

Gate: ruff check, ruff format --check, mypy (230 files), mkdocs build --strict clean; 2739 passed.

One pre-existing failure is unrelated and untouched: test_save_read_only_storage_returns_actionable_conflict chmods a directory read-only, which root ignores. It fails identically on a clean checkout of the branch point and passes on CI's non-root runner.

Notes for the reviewer

  • Cross-worker replay is deliberately not implemented: it needs a broker, and shipping a Redis client would be a first-party vendor connector, which CLAUDE.md rules out. register_event_log_provider() is the extension point, and the docs state the single-process limitation plainly.
  • The model probe adds a ~1.5s worst-case delay to init when no local server is running. AGENTOMATIC_LOCAL_MODEL skips it.
  • This branch twice hit CI failures from pre-existing code that surfaced only because the gate runs pre-commit over changed files (a ruff-format disagreement and a UP038 error). Root cause: .pre-commit-config.yaml pins ruff v0.8.6 while the ruff>=0.8 dev dependency resolves to 0.15.x. Worth aligning separately — bumping the pin here would reformat the repo and bury this change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V

…tory

`agentomatic init` resolved `agents/` against the current working
directory. Run it from a subdirectory of a project — or from the parent
it had just been created in — and it silently built a second, parallel
project, leaving the real `agents/` empty while reporting success. The
`--dir` flag made it worse: the bootstrap wrote `stacks/`, `main.py` and
`.agentomatic-stack` into whatever directory happened to contain the
target, a home directory in the worst case.

- Resolve the enclosing project by walking up from the current directory,
  stopping at a repository root or the user's home. Markers are specific
  (`.agentomatic-stack`, or `agents/` alongside `stacks/` or a platform
  `main.py`) so a scaffold cannot escape into a lookalike ancestor.
- Use a lone project found in a subdirectory (the `new` then forgot-to-cd
  case) and say so; name the candidates and stay put when several match.
- Add `--here` to opt out of detection.
- Confine `--dir` to placing the package: bootstrap only when the target
  actually sits inside an identifiable project.
- Fix the same cwd assumption in `add connection` (failed outright from a
  subdirectory) and `add ingestion` (landed in `<cwd>/ingestion/`).

Also, so that a scaffolded project runs and deploys as advertised:

- Prompt for NAME instead of hard-exiting on a bare `agentomatic init`,
  and reject names that are not importable package names.
- Report the absolute location and verify files reached disk, rather than
  claiming success over an empty directory.
- Bootstrap the platform `main.py` (never overwriting an existing one, and
  warning when it is not an Agentomatic app) — `deploy` renders
  `uvicorn main:app`, so a project without one built a container that
  could not boot. Keep the bootstrap narrow: no empty `plugins/`,
  `endpoints/`, `ingestion/` or `pipelines/`.
- Detect the model a local OpenAI-compatible server has loaded and bake
  that into `stacks/local.yaml`. The hardcoded default meant any other
  loaded model answered the first invoke with HTTP 400.
- Add `openai` (langchain-openai) to the `all` extra: the default local
  stack's `omlx` provider needs it, so `agentomatic[all]` scaffolded a
  project whose every LLM call failed on a missing client library.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
The PR Quality Gate runs pre-commit over the files a PR changes, and its
ruff-format hook is pinned to v0.8.6 while the repo's dev dependency
resolves to ruff 0.15.17. The two wrap an assert message in opposite
directions, so this pre-existing block reformatted under the pinned hook
and failed the gate as soon as a PR touched the file — it is unrelated to
the surrounding change and already reformats on main.

Hoist the message into a local so neither version needs to wrap it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
…nection

Task events existed only while somebody was listening: ``_emit`` returned
early when a task had no subscribers, so everything emitted while a client
was disconnected was never created, let alone retained. A dropped SSE
connection could therefore only be recovered as far as the task's *current*
state — every intermediate step was lost, and there was no way to ask for
the gap.

- Give TaskEvent a monotonic per-task ``sequence``, gapless from 1.
- Add TaskEventLog: an ordered, bounded replay buffer with an in-memory
  default, a null implementation, and a provider registry so a shared
  backend can be registered for cross-worker replay (the interface ships,
  not a vendor client).
- Record every event whether or not anyone is subscribed.
- Publish the sequence as the SSE ``id:``, so a browser EventSource resumes
  through ``Last-Event-ID`` with no client code; ``?since=`` does the same
  for other clients. A new subscriber's snapshot also carries a cursor, so a
  client that drops immediately after it can still resume precisely.
- Announce a truncated resume point instead of serving a partial history as
  if it were complete.
- Drop a task's retained events when the task record is deleted.

This lands for every target type at once — agents, pipelines, plugins,
endpoints and ingestion share the one task manager, so a pipeline's per-step
progress now resumes exactly like an agent's.

A2A gains the matching surface: GET /a2a/tasks/{id}/events streams the same
events shaped as A2A task objects, with the same resumption contract. The
agent card stops hardcoding its capabilities and reports
streaming/pushNotifications/stateTransitionHistory from what the deployment
actually wired up — a card claiming a capability the platform lacks sends
clients down a path that then 501s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
The task stream became resumable, but the other streams still wrote frames
straight to the socket and kept nothing. A client that dropped mid-answer on
``/invoke/stream`` lost the partial response entirely, with no id to
reconnect against and nothing to reconnect to.

Add agentomatic.streaming: one SSE encoder, a stream identity, and a bounded
replay buffer with the same per-stream and cross-stream retention caps as the
task event log.

- ``/invoke/stream`` numbers every frame, records it, and returns the
  identity as ``X-Stream-Id``.
- ``GET /invoke/stream/{stream_id}`` re-sends the frames already produced,
  honouring ``?since=`` and ``Last-Event-ID`` and announcing a truncated
  resume point rather than serving a partial history as a complete one.
- Studio's run and resume streams are numbered through the same helper, so
  a reloaded debug UI is not forced to re-run an expensive agent.

Replay here recovers what was *produced*; it does not restart a run, because
the run is bound to the request that was cancelled. The docs say so plainly
and point at the durable path — submit as a task, follow the task events —
rather than implying a guarantee this cannot make.

Also fix a pre-existing UP038 in router_factory that the pinned ruff 0.8.6 in
.pre-commit-config.yaml flags but the repo's own ruff 0.15 no longer does. It
already fails on main and blocked the gate as soon as a PR touched the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
@piotrlaczkowski piotrlaczkowski changed the title fix(cli): scaffold into the real project instead of the current directory fix(cli): scaffold into the real project, and make streams resumable Sep 9, 2026
@piotrlaczkowski piotrlaczkowski changed the title fix(cli): scaffold into the real project, and make streams resumable feat(platform): fix init targeting and make SSE streams resumable Sep 9, 2026
The card was built in two places and they had drifted.
``/.well-known/agent.json`` — the canonical A2A discovery document, and the
one a conforming client actually reads — emitted a reduced card: no
capabilities block at all, and only ``invoke`` and ``chat`` in endpoints. A
client doing proper discovery therefore learned *less* about an agent than
one that guessed the per-agent ``/card`` URL, and could not discover the
streaming or A2A task endpoints at all.

Both now render through core.agent_card.build_agent_card, so a capability
can only be advertised in one place and the two cannot drift again. The
discovery card gains framework, capabilities, metadata and the full endpoint
set; its platform envelope is unchanged.

The endpoint list also names what the earlier commits added — the A2A task
stream and the invoke-stream replay route — so they are discoverable rather
than only documented.

Also let create_streaming_response opt into the same treatment: pass a
``stream_id`` and a user-built endpoint gets numbered, retained frames and an
``X-Stream-Id`` header, exactly like the built-in streams. Omitting it keeps
the previous behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
Studio builds its SSE frames inside run_tracker, so the numbering is applied
by wrapping the generator at the router rather than at the yield sites. That
is easy to undo by accident: a refactor that stops wrapping would drop the
ids silently, with every test still green.

Cover both the sequence and the retention under the run id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
…config

Production hardening of the replay work. Three defects and one gap.

**Retention was not a memory bound.** Both buffers capped entries by count
only: 512 frames across 256 streams, and 512 events across 1024 tasks. With
frames carrying generated answers that is several hundred megabytes of
entirely ordinary traffic, and a long answer per frame pushes it past a
container's limit. Both now carry byte budgets alongside the counts, with
per-stream and total ceilings, and the deques lost their ``maxlen`` because an
implicit drop evicted an entry without its size leaving the running total —
the accounting would have drifted upwards until nothing could be retained.
The newest entry always survives, so a single oversized frame is still
replayable rather than silently lost.

**Studio keyed its resume stream on ``thread_id``** — a client-supplied path
parameter. Resuming the same thread twice continued the first stream's
numbering, so a client reconnecting with Last-Event-ID was served the wrong
frames, and a caller could reach another thread's retained frames by naming
it. Keyed on a fresh per-request id instead.

**The A2A stream read the task record once per replayed event**, though the
record is only needed for the terminal frame — invisible with the in-memory
store, one query per frame with SQL. Read once.

**Retention is now sized from the environment** like every other platform
switch, so an operator can cap or disable it without editing Python:
AGENTOMATIC_TASK_EVENT_LOG / _EVENTS_PER_TASK / _EVENT_TASKS / _EVENT_MAX_MB
and AGENTOMATIC_STREAM_REPLAY / _FRAMES / _COUNT / _MAX_MB. A bad value logs
and falls back rather than stopping the platform booting.

Also confirms with tests that every new streaming route sits behind auth.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
The event log and the stream replay buffer both hold memory now, sized by
limits an operator is expected to tune. Without visibility there is no way to
tell whether those limits fit the box: the buffers were invisible until
something ran out of memory.

``/status`` gains a ``streams`` section and ``tasks.event_log``, each
reporting retained entries and bytes — ``bytes`` being the number to watch
against the configured ceiling.

``stats`` is deliberately not added to the TaskEventLog contract, so a
registered custom backend need not implement it; the field is simply absent
when it does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
A security review of this branch found a cross-agent authorization bypass in
the replay route added earlier in the PR, and reproduced it.

``GET /{agent}/invoke/stream/{stream_id}`` resolved the id against a single
process-wide buffer with no check that the retained stream belonged to the
agent named in the path. Zero-trust derives its policy purely from that agent
segment, so a caller permitted on a permissive agent could name a restricted
agent's stream id on their own agent's route and receive the restricted
agent's response frames — precisely the boundary the policy engine exists to
enforce. Studio run frames shared the same flat namespace, and Studio run ids
are enumerable through a path zero-trust does not cover at all.

The codebase already had the right pattern two files over: the Studio run
endpoint 404s when ``run.agent_name != name``. This restores it.

- Frames are recorded under an ownership tag pinning them to the agent that
  produced them and the principal they were produced for; reads under a
  different tag return nothing.
- The replay route answers 404 for a stream it may not read — the same answer
  as an id that does not exist, so the response never confirms someone
  else's stream is there.
- Studio frames are tagged ``studio:<agent>``, so a Studio run id is not a
  valid argument to an agent replay route.
- ``create_streaming_response`` takes an ``owner`` and its docstring now
  requires a fresh ``new_stream_id`` rather than suggesting one, so a
  third-party endpoint cannot key retention on a caller-supplied value.
- ``since`` rejects negatives at the edge on both new routes, matching the
  constraint the task route already applied.

Covered by tests that reproduce the original bypass and assert the
indistinguishable 404.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk5bxkw7Q3j8VN1TvxeQ5V
@piotrlaczkowski
piotrlaczkowski merged commit 59cf2e2 into main Sep 9, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants