Skip to content

fix(reflect): guard the condense swap against concurrent memory appends - #842

Open
HsienW wants to merge 3 commits into
HarnessMD:mainfrom
HsienW:fix/condense-swap-guard
Open

HsienW wants to merge 3 commits into
HarnessMD:mainfrom
HsienW:fix/condense-swap-guard

Conversation

@HsienW

@HsienW HsienW commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

What & why

When the reflector condenses an agent's memory.md, it reads the file, waits on the headless summary call (up to 180 s), rebuilds the file from what it read, and renames the result over the original. It never looks at the file again before the rename, so anything an agent appends during that wait gets silently overwritten. There's no error, no condense-abort line, and the lost text isn't in hive/backups/ either, since the backup is taken before the wait. Agents are told to append to memory.md as they work, and the reflector runs on a timer with no idle check, so this can happen in normal use.

This PR makes the swap check the file one more time right before the rename. If the file still starts with exactly the bytes that were condensed, any new bytes after them (an agent's append) get copied onto the end of the rebuilt file and kept. If anything else changed, the pass aborts with source-changed, the agent's file is left alone, and the next tick tries again from the new contents. Only src/main/reflect.ts changes, plus a new test file.

Where this fits. I'm planning a few small PRs for the condenser, and this is step 1. It only covers the input side: never swap over a file that changed while we were waiting on the summary. It doesn't try to fix everything in #501 or #744 on its own. Roughly in order:

  1. Step 1 (this PR), input side: check the file before the swap and carry appends over.
  2. Step 2, small hardening: log a reason instead of silently skipping an agent, catch errors at the tick level, clean up leftover temp files, and add a configurable size cap for memory.md.
  3. Step 3, output side ([BUG] Munder Difflin v0.5.2 — condense subsystem kills the app during idle compaction + corrupts agent notebooks #501 Bug 2): pin each hidden run to its own transcript and have the reply carry a pass id, so condense can never pick up another session's answer.
  4. Step 4, archive before trim (Fix the condenser order #744): keep the evicted text somewhere durable before it's dropped. I'd love your take on how 0.5.5 handles memory-archive-raw before I start on this one.

Each one is its own PR to keep reviews small, and this PR is useful on its own without the others.

Refs #501, Refs #744

What the swap does in each case
What the re-read finds What happens Log
exactly the snapshot rename, same as today condense
snapshot plus new bytes at the end (an append) copy the new bytes onto the temp file, fsync, check again (up to 3 rounds) condense with carriedBytes
any other change (in-place edit, rewrite, truncation, a carried tail that changed) abort, remove the temp file, keep the agent's file condense-abort reason=source-changed
still growing after 3 rounds same as above source-changed, detail still being appended
file missing or unreadable abort, never recreate the file from the snapshot source-check-failed
write or rename throws (for example EPERM on Windows) abort, remove the temp file swap-failed, same as today

A few details:

  • The snapshot is kept as raw bytes and the whole prefix is compared, so an equal-length edit with the mtime restored is still caught. The carried tail is joined as raw bytes too, so a multi-byte UTF-8 character split at the snapshot boundary comes through intact.
  • A source that isn't valid UTF-8 is refused before the LLM call (source-encoding-invalid) instead of being rewritten through a lossy decode.
  • The log only gains one optional field (carriedBytes) and a few new reason values. Memory format, agent protocol, IPC, and config are unchanged.

Known limit: this is an optimistic check, not a cross-process compare-and-swap. The last re-read and the rename run back to back with no await between them, so the window shrinks from the whole LLM call to a few milliseconds, but a write that lands in exactly that gap can still be lost. On POSIX, a writer that opened its fd before the rename keeps writing to the old inode. Closing that gap completely would need a single writer per file, which felt like a bigger design call than this PR should make.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Docs
  • Build / CI

Evidence

Before

On main, the new tests go red. 14 of the 17 fail, and the first one is the bug itself: an append made while the summarizer is pending doesn't survive the condense.

2026-11-11082331
$ node --test test/reflect-source-change.test.cjs   # on main
not ok 2 - a write appended while the summarizer is pending is carried onto the condensed file
  error: 'the concurrent append must survive, verbatim and last'
not ok 10 - an in-place edit while the summarizer is pending aborts and keeps the edit
not ok 12 - a source deleted while pending is not resurrected from the stale snapshot
...
# tests 17
# pass 3
# fail 14

I also ran a stress test on Windows (mock LLM, real NTFS, 300 condense passes, with a second process appending every 20 ms, which is far more often than a real agent writes). On main, every append made during the LLM wait was lost:

appends during the LLM wait lost external lines lost swap-failed (EPERM)
main 300 / 300 632 0

After

Same 17 tests on this branch: all green.

2026-11-11082358
$ node --test test/reflect-source-change.test.cjs   # on this branch ef781308
# tests 17
# pass 17
# fail 0

Same stress test on this branch:

appends during the LLM wait lost external lines lost swap-failed (EPERM)
this branch 0 6 to 10 (about 1.5%), all in condensed passes 0.3% to 1.7%, about 4% with a concurrent reader

The remaining external-line loss is the few-millisecond window from the known limit above; none of it happened in aborted passes. Every swap-failed left the file untouched with the temp file removed, and the next tick retried.

And in the real dev app with a real hidden Claude run. I seeded the god agent's memory.md with 30 sections, triggered await window.cth.reflectNow('god') from DevTools, and had a script change the file while the summary call was pending:

image
Run What changed during the call Result
append a marker line appended every 3 s condense with carriedBytes: 418 (11681 to 3992 bytes). All 11 markers are still in the file afterwards (3992 + 418 = 4410 bytes), and no temp file was left behind.
in-place edit the heading of note 1 rewritten after the backup was taken condense-abort reason=source-changed. memory.md kept the edit (11543 bytes, not overwritten), and no temp file was left behind.

How I tested it

  • OS: Windows 11, Node 22.22.3
  • Steps:
    1. node --test test/reflect-*.test.cjs: 24/24 pass (17 new plus the 7 existing reflect-summary tests). The new file fails 14/17 on main, and 7/17 against an abort-only version of the fix, so the tests cover the carry-over path too, not just the abort.
    2. npm run typecheck (node and web): pass.
    3. npm run test:focused on this branch and on main 9bc1e5a: 944 pass / 24 fail vs. 927 pass / 24 fail. The extra 17 passes are the new tests. The 24 failures are the same list on both sides, all Windows-only symlink, worktree, and electron-builder tests that have nothing to do with reflect.
    4. npm run build: succeeds.
    5. The Windows stress test and the dev-app runs described under Evidence. One earlier dev-app attempt ended in summarize-failed because the hidden run never wrote a transcript, so the swap never ran. A retry went through fine. That failure is on the [BUG] Munder Difflin v0.5.2 — condense subsystem kills the app during idle compaction + corrupts agent notebooks #501 Bug 2 path (a hidden run that writes nothing), which step 3 will cover; this change doesn't cause it.

The second commit is the new test file. The third removes a stray 0 before the summary-marker doc comment that I left behind in #508 (it parsed as a harmless 0; statement).

Heads up: #671 also touches reflect.ts. I checked a merge against its current head and it goes through without conflicts.

Checklist

  • Before and after evidence is attached above, under both headings.
  • npm run typecheck passes.
  • npm run test:focused passes. (No new failures: the same 24 Windows-only failures show up on main.)
  • npm run build succeeds.
  • This PR is one change. Unrelated fixes belong in their own PR.
  • I read the diff myself before opening this, and there is no debug output,
    commented-out code, or unrelated formatting churn in it.
  • Any new UI derives from DESIGN.md / tokens.ts — no ad-hoc colors,
    spacing, or fonts. (No UI changes.)
  • If I added art, it's my own or compatibly licensed, and listed in
    ATTRIBUTION.md. (No art added.)

- Re-read memory.md as raw bytes right before the rename and compare it with the condensed snapshot.
- Carry an agent's appended tail over verbatim, re-checking up to three rounds.
- Abort on any other change with source-changed, and on a missing or unreadable file with source-check-failed.
- Refuse a non-UTF-8 source before the summary call with source-encoding-invalid.
- Remove the temp file on every failure path and log carriedBytes on success.
- Gate the summarizer so tests can mutate memory.md while the call is pending.
- Cover carried appends, multi-byte boundaries, in-place edits, deletion and rename failures.
- Assert the temp file is cleaned up and the carried tail survives the next pass.
- Remove the literal 0 that HarnessMD#508 left in front of the doc comment.
- Keep behavior unchanged, since ASI already made it a no-op statement.
aaroncoville added a commit to aaroncoville/munder-difflin that referenced this pull request Nov 12, 2026
Condensing reads memory.md, waits on a summary call that can take minutes,
then renames the rebuilt file over the original. Anything an agent appended
in between was overwritten by the rename and lost.

The swap now re-reads memory.md as raw bytes just before the rename and
compares it with the snapshot the summary was built from:
- bytes appended after the snapshot are carried onto the new file verbatim,
  re-checking up to three rounds while the file keeps growing;
- any other change aborts with source-changed (the next pass retries from
  the new file), and a missing or unreadable file aborts with
  source-check-failed;
- a source that is not valid UTF-8 is refused before the summary call
  (source-encoding-invalid), since a lossy decode would corrupt it;
- the temp file is removed on every failure path, and a successful swap logs
  carriedBytes.

The not-smaller back-off on the autonomous loop is unchanged.

Ported from upstream PR HarnessMD#842 (commits 9a560db,
6143a40, ef78130).

Co-authored-by: Aaron Coville <aaron@woodenbird.io>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant