Stream live span operations directly to routes and reclaim delivered payloads

bt-daemon/Cargo.toml

@@ -33,7 +33,7 @@ fs2 = "0.4"
 regex = "1"
 rquickjs = "0.12.2"
 rquickjs-serde = "0.6.1"
-serde = { version = "1", features = ["derive"] }
+serde = { version = "1", features = ["derive", "rc"] }
 serde_json = "1"
 semver = "1"
 sha2 = "0.10"

bt-daemon/README.md

@@ -305,49 +305,27 @@ different profiles, organizations, projects, experiments, or parent spans.
 
 ## Recovery and storage
 
-Capture requests return after the event is flushed to its journal. Daemon
-workers handle authentication, translation, and delivery. Translation commits
-canonical SpanOps and its consumed event position to a SQLite journal. Each route
-has an independent durable head that advances only after checked sink acceptance
-or intentional filtering. Plugin, authentication, and delivery recovery resumes
-from unresolved SpanOps without replaying translation. Retries use deterministic
-span IDs, so repeated delivery updates the same spans.
-
-Provisional session roots stay hidden until their first child arrives. Root
-candidates are saved with translation's input checkpoint, independently of route
-delivery, so an unused root does not hold completed turn batches open. When work
-arrives, the root's saved operations are published before its child.
-
-Failures retain their reason and retry condition, including across restarts.
-The daemon waits for the relevant plugin/configuration change, fresh credentials,
-or retry delay before trying again. `bt trace doctor` reports the affected stage,
-pending position, reason, and retry eligibility.
-
-A later event with corrected delivery settings also resumes configuration-held
-subscriptions for the same source session, authentication selection, and
-destination. Each subscription keeps its original durable head and pinned backend;
-the correction is saved for restart recovery. Healthy routes and other destinations
-retain their own settings.
-
-Claude, Codex, and Cursor events reference daemon-owned transcript mirrors. Cursor
-preserves separate immutable generations when its native transcript is rewritten.
-Grok uses
-separate updates and events mirrors. Replay does not depend on the original
-transcript staying at its old path.
-
-The daemon streams journals and transcripts, applies queue backpressure, and
-retires idle sessions. Retirement preserves unresolved work and route heads;
-startup resumes pending subscriptions without waiting for another event.
-Translators rebuild their in-memory state from committed input when more
-translation is needed, suppressing already committed output. Journals and mirrors
-preserve conversation content; credentials are excluded from journaled routing data.
-
-Completed, inactive session partitions become eligible for cleanup after seven
-days without modification. Cleanup also requires translation to be caught up,
-all captured routes to have registered heads, every head to be complete, and no
-pending recovery. Retained event logs protect the transcript mirrors and
-correlation state needed for reconstruction. Cleanup runs at startup and hourly;
-the retention period is fixed in `src/server/maintenance.rs`.
+The daemon saves captured events locally before acknowledging them, then uploads
+traces in the background. Pending work resumes after a daemon restart without
+requiring another event from the agent. Retries update the same spans rather than
+creating duplicate traces.
+
+Authentication, configuration, or plugin failures can delay uploads. The daemon
+retries when credentials or settings change, or after a retry delay for temporary
+failures. Run `bt trace doctor` to see what needs attention. Delivery to each
+destination recovers independently. To send older, completed session history to a
+new destination, use transcript import.
+
+Uploads are batched for up to 30 seconds, with earlier uploads when the batch
+reaches its size limit. Turn endings, explicit flushes, and daemon shutdown drain
+available traces without waiting for that interval.
+
+Local event logs and transcript copies contain conversation content. Credentials
+are excluded from saved routing settings. Storage can grow as sessions accumulate
+messages or uploads remain pending. Space used by completed delivery records is
+reused, so a database file may keep its previous size after delivery catches up.
+Completed sessions become eligible for cleanup after seven days without changes;
+unfinished work is retained for recovery.
 
 ## Platform and integration coverage
 

bt-daemon/src/correlation.rs

@@ -311,10 +311,9 @@ impl CorrelationRegistry {
             }
             if row.end_ms.is_some() {
                 if let Some(tools) = state.active_tools.get_mut(key) {
-                    if let Some(tool) = tools.get_mut(&row.span_id) {
-                        tool.active = false;
-                        changed = true;
-                    }
+                    // Completed tools can no longer parent a child session.
+                    // Duplicate terminal operations are harmless after removal.
+                    changed |= tools.remove(&row.span_id).is_some();
                 }
                 continue;
             }
@@ -731,6 +730,25 @@ mod tests {
         );
     }
 
+    #[test]
+    fn completed_tools_release_their_entries_without_retiring_the_session() {
+        let registry = parent_registry();
+        for i in 0..10_000 {
+            let id = format!("tool-{i}");
+            open_span(&registry, "parent", &id, SpanType::Tool);
+            close_span(&registry, "parent", &id);
+            close_span(&registry, "parent", &id);
+            assert!(registry.state.lock().active_tools["parent"].is_empty());
+        }
+        assert!(registry
+            .active_parent_snapshot("parent")
+            .unwrap()
+            .tools
+            .is_empty());
+        open_span(&registry, "parent", "next-tool", SpanType::Tool);
+        assert!(registry.has_active_tools("parent"));
+    }
+
     #[test]
     fn parent_registry_remains_usable_after_a_reader_unwinds() {
         let registry = parent_registry();