T9: daemon backfill sweep driver on the existing tick loop - #568
Merged
philcunliffe merged 1 commit intoAug 1, 2026
Merged
Conversation
… loop
New `src/core/daemon/backfill_sweep.js`. `createBackfillSweepDriver({backfills,
backfillMaterializers, env, config, storage})`'s `tick({now})` walks
`backfills.list()`, skips any contribution with no `sweep` field or a cron that
is not due (`cronMatches`, the sink driver's own due-check), and fires
`runBackfillProvider` per due contribution with a `sweep-<name>-<now>` dev run
id. Runs are fired unblocked: `tick()` resolves once each run has been started,
never once one finishes, so a provider's transcript scan cannot stall the sink
snapshots, the source-detail refresh, or `persist()` later in the same tick. Both
settlements are handled, so a failing run is a logged `backfill.sweep_failed`
record (component `openclaw`, operation `backfill.sweep`, `error_kind`) rather
than an unhandled rejection that would take the daemon process down. A malformed
`sweep.cron` is logged and treated as not due rather than thrown, so one
provider's bad metadata cannot skip the rest of the list.
`runtime.js`'s `runTick()` calls `await sweepDriver.tick({now})` directly after
the existing sink-driver tick, riding the same `DEFAULT_TICK_INTERVAL_MS`
60-second loop: a `*/5 * * * *` schedule only needs a due-check once a minute, so
this opens no second timer to start, drain, and account for at shutdown.
Also repairs the typecheck this task's branch point already failed: T7's
`sweep: { cron: opts.config?.backfill?.sweep_cron ?? ... }` does not compile,
because the plugin's config slice is a `JsonObject` and every step below its root
is a `JsonValue`. Read through a `resolveSweepCron` mirroring the
`resolveQuiesceMs` helper already sitting beside it; behavior is unchanged.
Externally blocked for real capture: until PR #552 (issue #543) merges, the
LLP 0158 reader still reads OpenClaw v3 fields flat, so a sweep projects nothing
from a real transcript. These tests passing is not evidence that it does.
Tests: the due-check fires only sweep-bearing, cron-due contributions and builds
the narrowed `BackfillRunnerContext` from the daemon's own runtime fields; a
rejected run neither throws out of `tick()` nor lands as an unhandled rejection,
and a never-settling run does not block the tick. A separate wiring test boots a
real daemon with a fixture plugin whose contribution opts into a sweep and proves
the tick actually runs it, which no unit test of the driver can show.
Task-Id: T9
philcunliffe
merged commit Aug 1, 2026
20f3a15
into
integration/openclaw-two-lane-capture
7 of 9 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements task T9 of openclaw-two-lane-capture.
New
src/core/daemon/backfill_sweep.js:createBackfillSweepDriver({backfills, backfillMaterializers, env, config, storage})whosetick({now})walksbackfills.list(), skips contributions with nosweepfield or a not-duecronMatches(imported fromsrc/core/sinks/driver.js), and firesrunBackfillProviderper due contribution with asweep-<name>-<now>dev run id. The fired promise's rejection is logged (componentopenclaw, operationbackfill.sweep,error_kind) rather than left unhandled.src/core/daemon/runtime.js'srunTick()callsawait sweepDriver.tick({now})right after the existing sink-driver tick, inside the sameDEFAULT_TICK_INTERVAL_MS = 60_000loop. No new timer.Also repairs the typecheck the branch point already failed on: T7's
opts.config?.backfill?.sweep_crondoes not compile against aJsonObjectconfig slice; it now reads through aresolveSweepCronmirroring theresolveQuiesceMshelper beside it, behavior unchanged.Externally blocked for real capture (PR #552): until the LLP 0158 reader projects OpenClaw v3's nested
messageenvelope, a sweep captures nothing from a real transcript. These tests passing is not evidence that it does.npm test(3326 pass, 0 fail),npm run typecheck, andnpm run smoke -- daemon_foreground_start_stopall pass.Task-Id: T9