Skip to content

fix(daemon): serve reset-required stores as a typed state - #2263

Merged
ScriptedAlchemy merged 4 commits into
masterfrom
fleet/reset-required-serving
Sep 27, 2026
Merged

ScriptedAlchemy merged 4 commits into
masterfrom
fleet/reset-required-serving

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

What was wrong

A daemon from origin/master exits at startup over a beta.54-era profile. #2202 moved Git correlation to schema 6. Every registered store that beta.53/54 wrote records schema 5: global.db (profile authority), user-sessions.db (profile sessions), and each project's sessions.db. Registered-schema admission refuses those stores, and two eager mounts at bootstrap turned that refusal into a process exit: the worker plan reads profile sessions, and the maintenance coordinator reads the profile authority. The account-deletion resume, the remote router, and connection identity binding each also returned the refusal with ?.

tracedecay update waits for the restored daemon to reach RunningEnabled, so it timed out and exited 1 with the daemon down. It never reached its own post-window reset.

What changed

Typed state. StoreResetRequiredV1 { store, authority, found_version, required_version, reason, remedy } lives in tracedecay-domain. DaemonSessionRuntimeRegistryV1 records it at the one admission point, attach_registered. A refused store stays unmounted, so every open re-runs admission, and a reset store serves on its next open. A successful attach clears the record. The Git correlation refusal is now the versioned ProfileResetRequired { component: "git correlation", found_version: Some(5), required_version: 6 }.

Daemon stays up. These paths now treat a reset-required store as a typed state instead of exiting:

  • The bootstrap worker plan runs on the Automatic selection, the value a reset profile initializes to, and logs profile_worker_plan_reset_required.
  • Maintenance resolves the profile authority on each tick. While it is refused, a tick logs retention_degraded and retries.
  • The account-deletion resume and the remote protocol router are skipped with typed log events.
  • Connection identity binds to the profile's registered global.db location, so requests get a typed answer instead of a closed socket.

Reads answer the typed problem.

  • Profile-retained reads (LCM, message search) answer reset_required with the reset legal action instead of a retryable unavailable.
  • The project-open JSON-RPC and MCP refusals handle the versioned variant.
  • Doctor's core runtime probe answers instead of closing the socket.

Stores that admit keep serving. When only a session store is refused, a project keeps its graph/search core. The retained core activates code indexing (full_upgrade_reset_required). The query-authority waiter now demands the complete generation itself: a restored generation seats only for a reader that asks for it, and a refused full upgrade has no advisory owner to ask.

Surfaces.

  • tracedecay_status (core and full) carries reset_required_stores, and markdown renders each as a pending operator action.
  • initialize _meta["tracedecay/reset_required_stores"] carries the same list.
  • doctor lists each store with its exact command. It exits 75 when it found no issue but a reset is pending (fix(cli): type host sweep outcomes and exit truthfully #2075's table).
  • The readiness probe parses the list into DaemonProtocolState::Ready { reset_required_stores }. The restore check accepts such a daemon as restored.
  • update no longer wipes the profile on the operator's behalf. It prints each pending reset and exits 75. The refusal remedy text drops the claim that update resets.

Fail before / pass after

typed_terminal_restart_acceptance::reset_required_serving::reset_required_profile_session_store_is_served_typed_until_its_named_reset runs a physically spawned daemon over an isolated profile. It stamps user-sessions.db to Git correlation v5 and restarts, then asserts:

  • the daemon reaches readiness;
  • tracedecay_status lists exactly [{"store":"profile sessions","authority":"git correlation","found_version":5,"required_version":6,"reason":"git correlation profile schema 5 is incompatible with required schema 6; reset the profile","remedy":"tracedecay wipe --all --yes"}];
  • tracedecay_lcm_status {storage_scope:user} returns kind: reset_required with legal_actions: ["reset"];
  • tracedecay_search on the same project still returns probe.

It then runs tracedecay wipe --all --yes, after which the same read serves and status lists [].

Same test binary, TRACEDECAY_TEST_BIN pointed at a build of origin/master fe6b7fd469:

tracedecay daemon exited before accepting connections: exit status: 1; stderr: Error: config error: git correlation persisted shape requires reset: the store records Git correlation schema version 5; this build stores Git evidence as per-session rows at version 6
test result: FAILED. 0 passed; 1 failed

With this branch: test result: ok. 1 passed; 0 failed.

Restore check: service::update_restore_tests::restore_check_accepts_a_reset_required_daemon_and_returns_its_pending_reset. A fake systemd unit reports RunningEnabled, and a socket answers initialize with the reset-required _meta. The test asserts restored_service_matches(...) == true and Ready { reset_required_stores: [<the literal store>] }. On master the probe drops _meta and has no pending-reset state to assert, so the test cannot be expressed there; the behavioral failure on master is the startup exit above. update_with_a_reset_required_store_is_pending_on_the_named_reset pins PendingOperatorAction and the literal printed line.

Journey on the built binary (real beta.54 profile)

  1. Created an isolated HOME with the verified v1.0.0-beta.54 x86_64-linux release binary, ran tracedecay init, and indexed a small repo. global.db, user-sessions.db, and projects/*/sessions.db all record git_correlation|5.
  2. Installed this branch's binary as the managed service. A fake systemctl on PATH runs the unit's ExecStart under a capped systemd-run scope.
$ tracedecay daemon status
state: running
service manager: RunningEnabled
socket: …/daemon.sock (connectable)
protocol: Ready

$ tracedecay doctor
  ✔ TraceDecay daemon unit is installed, enabled, and running.
  ! Store profile authority requires reset (git correlation profile schema 5 is incompatible with required schema 6; reset the profile). Pending operator action: run `tracedecay wipe --all --yes`
  ! Store profile sessions requires reset (…). Pending operator action: run `tracedecay wipe --all --yes`
  ! Current project is not served: git correlation profile schema requires reset (…). Pending operator action: run `tracedecay wipe --all --yes`
13 warning(s), no issues.
doctor exit=75

$ tracedecay update --no-reinstall
✔ Already up to date (v1.0.0-beta.54).
✔ TraceDecay writers stopped; exclusive maintenance window active.
✔ Daemon service refreshed at …/tracedecay.service
  pending operator action: profile authority requires reset (git correlation profile schema 5 is incompatible with required schema 6; reset the profile); run `tracedecay wipe --all --yes`
The TraceDecay binary and daemon are up to date; the daemon serves the stores listed above in their reset-required state until the operator runs the named reset.
update exit=75
systemctl calls: …stop tracedecay.service; …daemon-reload; enable; start tracedecay.service; is-active; is-enabled…

$ tracedecay wipe --all --yes          (stops and restarts the managed service itself)
Wiped complete profile database state (8 filesystem entries).
wipe exit=0

$ tracedecay daemon status  → state: running, protocol: Ready
$ tracedecay tool tracedecay_lcm_status {storage_scope:user}  → problem: None | status: ok
$ tracedecay init; tracedecay tool tracedecay_search alpha  → freshness: fresh, **alpha** (function) rank 1
$ tracedecay_status reset_required_stores → []
$ tracedecay doctor → 9 warning(s), no issues. exit=0

On a real beta.54 profile, global.db is refused too, so project reads return the typed refusal until the reset. The "code index still answers" case applies when only session stores are refused, which is what the integration test covers.

Suites (post-rebase on e3adbfba17, run under a capped systemd-run scope)

  • tracedecay-daemon-control lib: 99 passed
  • tracedecay-daemon-service lib: 320 passed; shutdown_coordination: 2 passed
  • tracedecay-cli bin (--features test-transport): 330 passed; host_journeys_suite: 35 passed
  • tracedecay-global-db lib: 374 passed; registered_schema_fail_closed: 1; session_relation_graph: 8
  • tracedecay-session-runtime lib: 122 passed
  • tracedecay-sessions lib: 549 passed
  • tracedecay-store-runtime lib: 122 passed (1 ignored)
  • tracedecay-domain lib: 222 passed; domain_suite: 171
  • tracedecay-mcp lib: 381 passed plus workflow::test_runner 10/10 run serially. Those 4 cargo-fixture tests time out only when run concurrently under load.
  • tracedecay transport_acceptance_suite typed_terminal_restart_acceptance: 5 passed
  • tracedecay lib: 758 of 759 passed. The failing test is daemon::tests::restart_proxy::serve_attaches_during_first_service_start_once_the_record_appears, which fails identically when run alone and exercises only files this PR does not touch (core_proxy.rs, daemon identity).
  • tracedecay mcp_suite: 571 of 572 passed with 4 threads. retrieve_truncation_test::diff_context_large_response_uses_retrievable_truncation_handle hit "interactive catalog is warming" under load and passes 3/3 alone. status_behavior_test now also expects the github_source line fix(configuration): provision the GitHub origin source binding #2221 added.
  • cargo clippy -p tracedecay-domain -p tracedecay-global-db -p tracedecay-store-runtime -p tracedecay-mcp -p tracedecay-daemon-protocol -p tracedecay-daemon-control -p tracedecay -p tracedecay-cli --all-targets -- -D warnings, with and without --features tracedecay-cli/test-transport,tracedecay/test-transport: clean.
  • cargo fmt --all -- --check: clean. pnpm run contracts:check: contracts up to date.

Windows: windows_task.rs only gains the Ready { .. } pattern; not run on Windows (#2081/#2082/#2083/#2105).

Fixes #2230

A registered store whose persisted shape this binary does not open no
longer stops the daemon. It stays unmounted in a typed reset-required
state that status, doctor, and initialize name with the exact reset
command; update's restore accepts such a daemon and exits 75 on the
named reset instead of wiping the profile itself.

Fixes #2230
@changeset-bot

changeset-bot Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: a6c721c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-26T20:23:12.643575Z 88560a6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88560a6a78

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1470 to +1473
Box::pin(attach_reset_required_stores(
response,
&engine.store_administration,
))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add reset metadata to every initialize route

The reset metadata is attached only in the Unix bootstrap branch. I checked the portable Windows broker and routed RMCP paths: the former writes its bootstrap response without this call, while the latter's decorator adds only route metadata. Because query_daemon_identity_stream treats an absent metadata key as an empty list, update and doctor on Windows can falsely report no pending reset, and cached-project initialize responses omit the advertised reset state on every platform. Apply the reset-store decoration to all successful initialize response paths.

AGENTS.md reference: AGENTS.md:L209-L212

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

update cannot reset a persisted shape the new daemon refuses at startup

1 participant