Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
261 changes: 261 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

10 changes: 8 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,15 +194,20 @@ Format references for the agent on-disk formats live at `docs/agents/formats/`

- `PathOrRef::Path` is `Box<Path>` to avoid a large enum variant size difference.
- Shared derivation: `toolpath-convo` provides the provider-agnostic `ConversationView → Path` mapping (`toolpath_convo::derive_path`). New conversation providers build on it rather than re-implementing the mapping.
- Provider extras: `Turn.extra` and `WatcherEvent::Progress.data` use provider-namespaced keys (e.g. `extra["claude"]`, `extra["gemini"]`) so trait-only consumers can reach provider metadata without importing provider types.
- Path kinds: `PathMeta.kind` is an optional URI naming a hosted kind spec; URIs are immutable and semver-versioned. The only one defined is `https://toolpath.net/kinds/agent-coding-session/v1.1.0` (`toolpath::v1::PATH_KIND_AGENT_CODING_SESSION`); every conversation → `Path` derivation sets it. Spec sources: `site/kinds/<name>/<version>/{index.md,schema.json}` (schema.json symlinks into `crates/path-cli/kinds/`, which `p validate` bundles). RFC section: "Document Kind".
- The conversation IR (`toolpath-convo`) is a **closed typed set** — no catch-all `extra`/passthrough fields on `Turn`, `Compaction`, or any other IR type; provider detail either earns a typed optional field or is accepted as lossy. Native-layer catch-alls (e.g. `toolpath-claude`'s `ConversationEntry.extra` serde-flatten) are fine — they represent the *source format*, not the IR. `WatcherEvent::Progress.data` (a live event stream, not the IR) still uses provider-namespaced keys (e.g. `data["claude"]`).
- Path kinds: `PathMeta.kind` is an optional URI naming a hosted kind spec; URIs are immutable and semver-versioned. The only kind defined is `agent-coding-session`, currently at `https://toolpath.net/kinds/agent-coding-session/v1.2.0` (`toolpath::v1::PATH_KIND_AGENT_CODING_SESSION`; `…_V1_1_0`/`…_V1_0_0` name the superseded URIs, still registered and bundled); v1.2.0 adds the `conversation.compact` step type on top of v1.1.0's message-level token accounting. Every conversation → `Path` derivation sets it. Spec sources: `site/kinds/<name>/<version>/{index.md,schema.json}` (schema.json symlinks into `crates/path-cli/kinds/`, which `p validate` bundles — all versions). RFC section: "Document Kind".

### Token accounting (kind v1.1.0)

- Two optional keys on `conversation.append`/`Turn`: `token_usage` is the total for a provider message, stamped on the message group's final step (`Σ` over a path = session total); `attributed_token_usage` is a step's own attributed spend, populated only where the source genuinely reports per-step spend. `Turn.group_id` groups the steps of one message (remainder = group total − Σ attributed, computed not stored).
- Hard rules: never stamp a cumulative counter, a repeated message total, or zero-filled placeholders onto a step. Claude's per-line `usage` is a cumulative streaming snapshot, not a per-block cost — take the field-wise-max group total and never derive attribution from it. Codex must difference the cumulative `total_token_usage`, never sum `last_token_usage` (re-emitted stale). pi/opencode decode all-zero wire counters as `None`.
- `breakdowns` is an optional decomposition of a top-level class into sub-classes (e.g. `breakdowns["output"]["reasoning"]`). Informational only — **never summed into any total**; invariant `Σ(inner) ≤ parent`; omitted when empty. Gemini and OpenCode fold reasoning tokens into `output_tokens` and record this breakdown (projectors un-fold on the reverse path); Codex differences `reasoning_output_tokens`; Claude reports no breakdown.

### Compaction (kind v1.2.0)

- The `conversation.compact` step type records a context-compaction boundary as its own step between the turns it separates (turns after the boundary parent on it, so the `head`-ancestry walk crosses it in order). `structural` fields are all optional but `type`: `trigger` (`auto`|`manual`), `summary`, `pre_tokens`, and `kept` (ids of prior turns surviving verbatim — always a contiguous parent-chain run, oldest first, whose first element is the anchor; empty = wholesale). It is not a turn (no `text`/`role`/`tool_uses`); `step.actor` is `tool:<provider>`.
- Compaction provenance is itself a closed typed set — no catch-all `extra`; native detail richer than the contiguous run (e.g. Claude's replay-pinned messages) is deliberately not carried, and round-trips are lossy beyond these fields. The IR carries the anchor as `Compaction.kept_from`; `toolpath_convo::expand_kept` expands it to the run. Steps whose parents were rewired by derive-splicing also carry `source_parent` (string|null) on their structural change. Each provider's `Item::Compaction` derivation populates it and projects it back to harness markers.

### Providers

- Data locations: claude `~/.claude/projects/` (JSONL); gemini `~/.gemini/tmp/<project>/chats/` (project slot is a friendly name or a path SHA-256; sub-agents in sibling UUID dirs); codex `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` (date-bucketed, not project-keyed); copilot `~/.copilot/session-state/<id>/events.jsonl` (`COPILOT_HOME` overrides; global, not project-keyed); opencode `~/.local/share/opencode/opencode.db` (read-only SQLite); cursor `state.vscdb` global SQLite under the Cursor app-support dir; pi `~/.pi/agent/sessions/` (tree in a single file, preserved as a DAG).
Expand All @@ -211,6 +216,7 @@ Format references for the agent on-disk formats live at `docs/agents/formats/`
- `toolpath-copilot` is a **preview** provider over a reverse-engineered `events.jsonl` schema; the reader is deliberately tolerant of shape variations and preserves unknown events. It is wired both directions: forward (`p import/list/show copilot`, `share`) and reverse via `CopilotProjector` (`p export copilot`, `resume` — writes `session-state/<id>/` plus a `session-store.db` row, only ever INSERTing a fresh id). The real `copilot --resume` loader is strict; its writer contract is documented in `docs/agents/formats/copilot-cli/writing-compatible.md` — follow it exactly.
- File-change fidelity varies: codex and copilot embed real diffs/file content in the session log, so derived paths get a true `raw` perspective; opencode diffs a sibling snapshot git repo via `git2`, falling back to structural-only changes for gitignored paths; cursor round-trips numeric tool-dispatch ids via `TOOL_TABLE` so projector-written composers render correctly in Cursor.app (the cursor-agent CLI's separate protobuf store is not parsed).
- `ConversationMetadata`/`SessionMeta` for claude/gemini/pi expose `first_user_message: Option<String>` (cheap, populated during the metadata pass) — used by pickers and any "list sessions by topic" surface.
- Compaction mappings: **claude** projects fully empty assistant turns as Claude's own empty thinking block (Claude 2.1.216's transcript renderer aborts on `[{"type":"text","text":""}]`). **codex** (2026-07 payload): `compacted.message` is empty — the summary lives encrypted in `replacement_history` and the surviving turns are a prefix-keep the suffix-anchored `kept_from` contract cannot represent, so the boundary derives wholesale with no summary; the projector re-emits the `event_msg`/`context_compacted` marker the TUI renders from, and places the opening `turn_context` before the first real prompt (codex titles a backfilled session from the first user message). **copilot** (observed at 1.0.68): a successful `session.compaction_complete` becomes the typed `Item::Compaction` (`summary` ← `summaryContent`, `pre_tokens` ← `preCompactionTokens`; wholesale — Copilot reports removed-message counts, not surviving ids); failed compactions and `session.compaction_start` stay generic events; the projector emits the observed pair back and mirrors the summary to `checkpoints/NNN-*.md`. **pi**: `model_change`/`thinking_level_change`/`label` entries map to typed `Item::Event`s (chain links — `expand_kept` passes through them) and project back; the projector threads the chain's model context into assistant `provider`/`api` so pi restores the resumed session's model (verified in pi 0.72); compaction `details`/`fromHook` are deliberate loss.

### CLI behaviors

Expand Down
10 changes: 5 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,18 @@ edition = "2024"
license = "Apache-2.0"

[workspace.dependencies]
toolpath = { version = "0.7.0", path = "crates/toolpath" }
toolpath = { version = "0.8.0", path = "crates/toolpath" }
toolpath-convo = { version = "0.12.0", path = "crates/toolpath-convo" }
toolpath-git = { version = "0.6.0", path = "crates/toolpath-git" }
toolpath-git = { version = "0.7.0", path = "crates/toolpath-git" }
toolpath-claude = { version = "0.13.0", path = "crates/toolpath-claude", default-features = false }
toolpath-gemini = { version = "0.7.0", path = "crates/toolpath-gemini", default-features = false }
toolpath-codex = { version = "0.7.0", path = "crates/toolpath-codex" }
toolpath-copilot = { version = "0.2.0", path = "crates/toolpath-copilot" }
toolpath-opencode = { version = "0.6.0", path = "crates/toolpath-opencode" }
toolpath-cursor = { version = "0.3.0", path = "crates/toolpath-cursor" }
toolpath-github = { version = "0.6.0", path = "crates/toolpath-github" }
toolpath-dot = { version = "0.5.0", path = "crates/toolpath-dot" }
toolpath-md = { version = "0.7.0", path = "crates/toolpath-md" }
toolpath-github = { version = "0.7.0", path = "crates/toolpath-github" }
toolpath-dot = { version = "0.6.0", path = "crates/toolpath-dot" }
toolpath-md = { version = "0.8.0", path = "crates/toolpath-md" }
toolpath-pi = { version = "0.7.0", path = "crates/toolpath-pi" }
path-cli = { version = "0.17.0", path = "crates/path-cli" }
pathbase-client = { version = "0.2.0", path = "crates/pathbase-client" }
Expand Down
14 changes: 10 additions & 4 deletions RFC.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,10 +296,16 @@ unrecognized URIs should be treated as a generic path. Kind URIs are
immutable, semver-versioned, and revisions ship at a new version URI.

Defined kinds are listed at <https://toolpath.net/kinds/>. The only one defined
so far is `https://toolpath.net/kinds/agent-coding-session/v1.0.0` — a path
recording an AI coding conversation, where each conversational-turn step
carries a `"conversation.append"` structural change with the turn's role,
text, and so on. See the linked spec for the full contract.
so far is `agent-coding-session`, currently at
`https://toolpath.net/kinds/agent-coding-session/v1.2.0` — a path recording an
AI coding conversation, where each conversational-turn step carries a
`"conversation.append"` structural change with the turn's role, text, and so
on, and context-compaction boundaries carry a `"conversation.compact"` step
whose `kept` list names the prior turns that survived into the post-compaction
window — always a contiguous parent-chain run, oldest first, anchored at its
first element. The earlier `v1.1.0` URI (message-level token accounting,
without `conversation.compact`) and `v1.0.0` URI both remain valid and
documented. See the linked spec for the full contract.

#### Actor Definitions

Expand Down
Loading
Loading