Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,109 @@

All notable changes to the Toolpath workspace are documented here.

## The actor grammar gets a type, and turns record their author — 2026-08-06

Every Toolpath step names an actor: `human:alex`, `agent:gpt-5.5`,
`tool:rustfmt`. That string is a grammar, and until now nothing owned it.
`toolpath-convo` formatted it with `format!`, parsed it back with a
hand-rolled reader, and other crates re-derived pieces of it with
`starts_with` — while `toolpath`, the crate that *defines* `step.actor`
and `meta.actors`, had no actor type at all.

Nothing enforced what the prefixes meant, and a real misattribution grew
in the gap. `toolpath-convo` built a turn's actor from its role plus a
model name: assistant plus a model name meant a model reply. That worked
until a harness wrote an assistant message itself — an API error, a
rate-limit notice, a timeout — with no model call behind it. Some
harnesses record a placeholder where the model name goes, and
`derive_path` took it at face value: the step landed on an `agent:` actor
naming a string that is not a model, and `meta.actors` described the
harness as if it were one. `agent:<model>` never meant that.

- **`toolpath`** (0.7.1): new `Actor` — the parsed `step.actor`. It is a
prefix and an id, each non-empty and drawn from `[A-Za-z0-9_.-]`, and
nothing else. The prefix set is **open**: `human`, `agent`, `tool` and
`ci` are conventions the format's users adopt, not a vocabulary the
format enforces, so `bot:dependabot` is as much an actor reference as
`human:alex` and this type gives none of them special meaning.
`Actor::new(prefix, id)` validates and returns `ParseActorError` on
anything the grammar cannot render back; `Display` and `FromStr` are
the only implementation of the grammar, and serde uses them, so an
`Actor` on the wire is the actor string. Every valid suffix-free
reference parses and renders back unchanged, and every `Actor` renders
a reference that parses back to itself. The `/`-delimited sub-actor
suffix (`agent:claude-code/tool:Write`, `tool:rustfmt/1.5.0`) is split
off and dropped on parse, as before; `Actor::split_sub_actor` exposes
the split for callers that need it. Additive: patch bump.
- **`toolpath-convo`** (0.12.0): `Turn.author` is an `Actor` and replaces
`Turn.model`. `role` keeps its own meaning: where the turn sits in the
conversation. The two were never the same question, and a harness
notice is the case that separates them — it occupies the assistant slot
without being model output.

A new `actor` module holds what the base format deliberately leaves
open: the `human:` / `agent:` / `tool:` prefixes this deriver uses,
the `human:user` placeholder it emits for an unnamed person, and the
`agent:unknown` placeholder the `agent-coding-session` kind spec
defines for "a model ran, unnamed". `agent:unknown` belongs to that
spec and `human:user` is this deriver's own habit — neither is a
property of the base grammar, so neither lives in `toolpath` any more.
Constructors (`generic_human`, `human`, `unnamed_agent`, `agent`,
`harness`) are total: a name the grammar cannot carry is no name at
all and falls back to the placeholder, which keeps every derived
actor string renderable and schema-valid whatever a session file
holds. Readers (`is_human`, `is_agent`, `is_tool`, `model_name`)
replace the prefix-specific accessors the base type used to carry.

`derive_path` now attributes a step by rendering its turn's author,
with no role matching and no string building of its own, and
`extract_conversation` recovers the author by parsing the actor back —
so derive → extract → derive is stable and the grammar lives in exactly
one place. Harness-authored turns take `tool:<provider>` — the actor
system turns and provider-specific roles already take — instead of an
`agent:` actor, which is the misattribution this fixes. `agent:unknown`
still means "a model ran, unnamed": distinct from "no model was
involved". Because a tool actor is always named, each provider now
supplies its own provider id when it builds a harness-authored turn,
rather than leaving the deriver to fill it in.

`meta.actors` records a `provider` only for the actors this derivation
mints itself. An actor the source supplied under some other prefix —
`ci:github-actions`, or anything else the open grammar allows — is
described by name alone: the deriver does not know where it came from,
and naming the harness there would assert provenance it cannot know.

One consequence of the open prefix set shows up on the way back in:
`extract_conversation` used to collapse any actor whose prefix was not
`human`, `agent` or `tool` to `agent:unknown`, because the type could
not hold it. It now reads such an actor back as itself, so a document
written by something with its own prefix survives derive → extract →
derive instead of being relabelled. Roles are unaffected — only the
three prefixes this crate attributes turns to map to a `Role`, and
anything else still lands in `Role::Other` carrying its reference.

Turns reach disk nested in a step's `delegations` payload, so `author`
also accepts the bare model name that older documents carry in its
place; the two are told apart by parsing, since a model name is not a
valid actor reference. Minor bump — replacing a public field on `Turn`
is breaking.
- **`toolpath-claude`** (0.13.0): maps Claude Code's placeholder model to
the harness's own tool actor and every other assistant message to an
agent actor, so the placeholder no longer reaches the IR. It is one
harness's format detail and stays owned by the crate that reads that
format; other providers set an author from whatever their own format
gives them. The projector writes the placeholder back when it
reserializes a harness-authored turn, so a session survives the round
trip. Minor bump — the crate now pins `toolpath-convo` 0.12.

No schema change: this is a deriver fix and a representation change,
valid under the existing `agent-coding-session` kind, and the base
schema's `actorRef` pattern is untouched. Derived documents are
byte-identical to those the previous release produced, apart from the
misattributed harness turns this fixes — checked by deriving every
checked-in harness fixture on both trees and diffing the canonicalized
JSON.

## Projected Claude sessions are resumable again — 2026-07-30

Two fixes found by live-resuming a projected session against the real
Expand Down
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,8 @@ Build the site after changes: `cd site && pnpm run build` (should produce 11 pag
- Gemini CLI conversation data lives in `~/.gemini/tmp/<project>/chats/`. Main sessions sit at the top (`session-<timestamp>-<short>.json`, `kind: "main"`); sub-agents live in sibling `<full-uuid>/` directories (`kind: "subagent"`). The `<project>` slot is either a friendly name from `~/.gemini/projects.json` or the SHA-256 hex of the absolute project path; `toolpath-gemini` resolves both.
- `toolpath-gemini` treats main file + sibling sub-agent UUID dir as one conversation. Sub-agent files are folded into `DelegatedWork` with populated `turns` (unlike `toolpath-claude`, whose sub-agent turns live in separate session files and stay empty). See `docs/agents/formats/gemini.md` for the full format reference.
- Provider-specific extras convention: `Turn.extra` and `WatcherEvent::Progress.data` use provider-namespaced keys (e.g. `extra["claude"]`, `extra["gemini"]`). `toolpath-claude` populates `Turn.extra["claude"]` from `ConversationEntry.extra`; `toolpath-gemini` populates `Turn.extra["gemini"]` with the full `tokens` struct, per-thought metadata, and tool-call status. This lets trait-only consumers access provider metadata without importing provider types.
- Actor grammar: `toolpath::v1::Actor` is the parsed `step.actor` — an opaque struct holding a prefix and an id, each non-empty and drawn from `[A-Za-z0-9_.-]`. The **prefix set is open**: `human`/`agent`/`tool`/`ci` are conventions, not a vocabulary, and the base crate privileges none of them (`bot:dependabot` parses). `Actor::new(prefix, id)` validates; `Display` + `FromStr` are the *only* implementation of the grammar and serde uses them, so an `Actor` on the wire is the actor string, and every valid suffix-free reference round-trips unchanged. A `/`-suffix (`agent:m/tool:Write`, `tool:rustfmt/1.5.0`) is a sub-actor qualifier: parsing keeps the segment before it and drops the suffix (`Actor::split_sub_actor` exposes the split). Build or read an actor through this type; don't `format!("agent:{}")` or `starts_with("human:")`.
- Actor conventions live in `toolpath_convo::actor`, not in the base crate: the `human:`/`agent:`/`tool:` prefixes, the `human:user` and `agent:unknown` placeholders (the latter defined by the agent-coding-session kind spec), constructors `generic_human`/`human`/`unnamed_agent`/`agent`/`harness` (all total — a name the grammar can't carry falls back to the placeholder) and readers `is_human`/`is_agent`/`is_tool`/`model_name`. `Turn.author` is an `Actor` and `derive_path` attributes a step by rendering it, so a harness-authored assistant message (the model placeholder Claude Code writes in that slot) lands on `tool:<provider>`, not a fake `agent:`. `toolpath-dot`, `toolpath-md`, path-cli's resume gate and `toolpath-git` still hand-roll prefix checks — a pending cleanup.
- Shared derivation: `toolpath-convo` provides a provider-agnostic `ConversationView → Path` mapping via `toolpath_convo::derive_path`. New conversation providers should build on it rather than re-implementing the mapping.
- Path kinds: `toolpath::v1::PathMeta.kind` is an optional URI naming a hosted kind spec; URIs are immutable and semver-versioned. The only one defined so far is `https://toolpath.net/kinds/agent-coding-session/v1.1.0` (constant `toolpath::v1::PATH_KIND_AGENT_CODING_SESSION`; `…_V1_0_0` names the superseded URI); every conversation → `Path` derivation sets it via the shared `toolpath_convo::derive_path` or each provider crate's own. Carried through the JSONL form via `PathOpen.meta` and `PathMeta` patch lines. Spec sources live in `site/kinds/<name>/<version>/{index.md,schema.json}` (schema.json is a symlink into `crates/path-cli/kinds/`, which `path p validate` bundles — both versions) and publish under `https://toolpath.net/kinds/`; the registry index is `site/kinds/index.md`. RFC: "Document Kind". JSON Schema: `$defs/pathMeta`.
- Token accounting (kind v1.1.0): two keys on `conversation.append`/`Turn`, both optional. `token_usage` = "the total for a message" (on the group's final step; `Σ` over a path = session total). `attributed_token_usage` = "this step's own attributed spend", populated only where the source genuinely reports per-step spend (its own key, so the sum is unaffected; remainder = group total − Σ attributed, computed not stored). One provider message can span several steps (Claude writes one JSONL line per content block); `Turn.group_id` groups them. `toolpath-claude` fills `group_id` from `message.id` and takes the **field-wise-max** group total (line order not trusted). Claude's per-line `usage` is a cumulative *streaming snapshot* (Anthropic streaming API: `message_start` seeds output near 0, `message_delta` is cumulative), NOT a per-block cost — so Claude emits no `attributed_token_usage`; the projector re-expands the total onto every line. `toolpath-codex` differences the cumulative `total_token_usage` (dedup-safe: never sum `last_token_usage` — Codex re-emits it stale; openai/codex #14489), attributes each per-call delta to the step it follows, and derives the round total from those attributions. pi/opencode decode all-zero wire counters as `None`. Never stamp a cumulative counter, a repeated message total, or zero-filled placeholders onto a step; never derive attribution from Claude's streaming snapshots.
Expand Down
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ edition = "2024"
license = "Apache-2.0"

[workspace.dependencies]
toolpath = { version = "0.7.0", path = "crates/toolpath" }
toolpath-convo = { version = "0.11.1", path = "crates/toolpath-convo" }
toolpath = { version = "0.7.1", path = "crates/toolpath" }
toolpath-convo = { version = "0.12.0", path = "crates/toolpath-convo" }
toolpath-git = { version = "0.6.0", path = "crates/toolpath-git" }
toolpath-claude = { version = "0.12.2", path = "crates/toolpath-claude", default-features = false }
toolpath-claude = { version = "0.13.0", path = "crates/toolpath-claude", default-features = false }
toolpath-gemini = { version = "0.6.1", path = "crates/toolpath-gemini", default-features = false }
toolpath-codex = { version = "0.6.1", path = "crates/toolpath-codex" }
toolpath-copilot = { version = "0.1.0", path = "crates/toolpath-copilot" }
Expand Down
10 changes: 5 additions & 5 deletions crates/path-cli/tests/cross_harness_matrix.rs
Original file line number Diff line number Diff line change
Expand Up @@ -767,18 +767,18 @@ mod invariants {
}
}

pub fn model_field(
pub fn author_field(
original: &ConversationView,
final_: &ConversationView,
failures: &mut Vec<String>,
) {
let o = meaningful_turns(original);
let f = meaningful_turns(final_);
for (i, (a, b)) in o.iter().zip(f.iter()).enumerate() {
if a.model != b.model {
if a.author != b.author {
failures.push(format!(
"model at turn {} diverged: first={:?} second={:?}",
i, a.model, b.model
"author at turn {} diverged: first={:?} second={:?}",
i, a.author, b.author
));
}
}
Expand Down Expand Up @@ -1011,7 +1011,7 @@ fn run_cell(
invariants::token_usage_survives(&view_after_source, &view_first, &mut failures);
invariants::thinking(&view_first, &view_second, &mut failures);
invariants::thinking_survives(&view_after_source, &view_first, &mut failures);
invariants::model_field(&view_first, &view_second, &mut failures);
invariants::author_field(&view_first, &view_second, &mut failures);
invariants::stop_reason(&view_first, &view_second, &mut failures);
invariants::parent_id_graph(&view_first, &view_second, &mut failures);
invariants::environment(&view_first, &view_second, &mut failures);
Expand Down
2 changes: 1 addition & 1 deletion crates/toolpath-claude/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "toolpath-claude"
version = "0.12.2"
version = "0.13.0"
edition.workspace = true
license.workspace = true
repository = "https://github.com/empathic/toolpath"
Expand Down
20 changes: 16 additions & 4 deletions crates/toolpath-claude/src/project.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@
//! reads a Claude JSONL conversation into a provider-agnostic view,
//! `ClaudeProjector` serializes that view back into the Claude wire format.

use crate::provider::SYNTHETIC_MODEL;
use crate::types::{
ContentPart, Conversation, ConversationEntry, Message, MessageContent, MessageRole,
ToolResultContent, Usage,
};
use serde_json::json;
use std::collections::HashMap;
use toolpath_convo::actor;
use toolpath_convo::{
ConversationProjector, ConversationView, ConvoError, Result, Role, ToolInvocation, Turn,
};
Expand Down Expand Up @@ -367,6 +369,16 @@ fn user_turn_to_entry(turn: &Turn, session_id: &str) -> ConversationEntry {
}
}

/// What belongs in `message.model` for an assistant entry: the model that
/// ran, or — for a message the harness produced itself — the placeholder
/// Claude Code writes in that slot, so a session survives the round trip.
fn model_field(turn: &Turn) -> Option<String> {
if actor::is_tool(&turn.author) {
return Some(SYNTHETIC_MODEL.to_string());
}
actor::model_name(&turn.author).map(str::to_string)
}

/// Build a `ConversationEntry` for an assistant turn. `wire_usage` is the
/// usage to write on the JSONL line: the IR carries a message's total only
/// on the group's final turn, but real Claude Code repeats `message.usage`
Expand Down Expand Up @@ -401,7 +413,7 @@ fn assistant_turn_to_entry_with_usage(
message: Some(Message {
role: MessageRole::Assistant,
content: Some(content),
model: turn.model.clone(),
model: model_field(turn),
id: turn.group_id.clone(),
message_type: None,
stop_reason: turn.stop_reason.clone(),
Expand Down Expand Up @@ -1041,7 +1053,7 @@ mod tests {
text: text.to_string(),
thinking: None,
tool_uses: vec![],
model: None,
author: actor::generic_human(),
stop_reason: None,
token_usage: None,
attributed_token_usage: None,
Expand All @@ -1061,7 +1073,7 @@ mod tests {
text: text.to_string(),
thinking: None,
tool_uses: vec![],
model: None,
author: actor::unnamed_agent(),
stop_reason: None,
token_usage: None,
attributed_token_usage: None,
Expand Down Expand Up @@ -1421,7 +1433,7 @@ mod tests {
#[test]
fn test_stop_reason_and_model_preserved() {
let mut turn = assistant_turn("a1", "Done.");
turn.model = Some("claude-opus-4-6".to_string());
turn.author = actor::agent(Some("claude-opus-4-6"));
turn.stop_reason = Some("end_turn".to_string());

let view = make_view("sess-1", vec![turn]);
Expand Down
Loading
Loading