Skip to content

A daily sweep takes a decaying source over its max_items back to it, lowest scores first - #21

Merged
gedaiu merged 1 commit into
re-devfrom
feat/source-decay-sweep
Oct 22, 2026
Merged

gedaiu merged 1 commit into
re-devfrom
feat/source-decay-sweep

Conversation

@gedaiu

@gedaiu gedaiu commented Oct 22, 2026

Copy link
Copy Markdown

Slice 4 of 5 of RC-003 (a source may decay), on top of #20. A source that sets decay is now kept to its max_items.

What is in it.

  • decay::sources_over_cap and decay::plan_eviction: for a source over max_items, take the 2 x excess oldest live documents that are at least min_age_days old, score them with Lore's score (document name as key, text length, retrievals, half-life from document_salience or the source, stale derived from the last retrieval), and pick the excess lowest. Equal scores keep the older first.
  • decay_sweep.rs: deletes each chosen document the way a person deletes one (so it can be restored), removes it from the full-text index, and writes one decay.evict audit row with no actor.
  • A daily ticker (copied from the governance one) queues one decay_sweep job per source over its cap with enqueue_unless_queued, and dispatch runs it. The job has the default priority, so it waits behind extraction.
  • docs/design/sources.md describes it.
  • Memory now owns its key and carries a text length, and is_stale is the one staleness rule.

Tests. A unit test for the selection and one for staleness; a store test builds a source with five documents (one too young, one recently read, one with a topic key) and pins which two go; a server test runs the sweep against a real database and search index and pins the deletion, the index removal, the audit row and that a second sweep does nothing.

Size. 527 lines, about 230 of them tests. Fine for the fork; to be split before it goes upstream.

Checked locally on the rebased branch: cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace (1688 passed, 0 failed).

🤖 Generated with Claude Code

…lowest scores first

Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>
@gedaiu
gedaiu force-pushed the feat/source-decay-sweep branch from c388bdb to 25f9472 Compare October 22, 2026 10:39
@gedaiu
gedaiu merged commit e550dab into re-dev Oct 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant