Repository navigation
A stale document of a decaying source ranks at 0.6 of its score and says it is stale - #23
Merged
Merged
Conversation
…ays it is stale Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice 5 of 5 of RC-003 (a source may decay), on top of #21. Search now uses what the earlier slices record.
What is in it.
decay::stale_documents: of the documents a search found, those that belong to a source withdecayset and have not been retrieved for the source'sstale_after_days(from the last retrieval, else from creation). Worked out in the database, never stored;confidenceis untouched.hybrid_scoredmultiplies the RRF score of a stale document's hit by 0.6 (STALE_SEARCH_WEIGHT, Lore's weight for a stale fact) and reorders before paging, so a stale document can drop off a page and not only move within it. The sort is stable.stale: true. The field is omitted when false, so a base where no source decays sees no change in its responses.docs/design/chat-and-mcp.mdand RC-003 say so.Cost. One extra query per search, keyed on the returned documents' ids, for every base (it returns nothing where no source decays).
Tests. A unit test pins the 0.6 factor and the reorder, and that a list with nothing stale keeps its order and scores; a store test pins which documents count as stale (only in a decaying source, past
stale_after_days, and not after a retrieval); a server test searches the same query before and after its source starts to decay and pins that the score is exactly 0.6 of the earlier one.Checked locally on the rebased branch:
cargo fmt --check,cargo clippy --workspace --all-targets -D warnings,cargo test --workspace(1692 passed, 1 failed:name_vectors::plan_tests::pending_reads_the_name_facts_through_the_predicate_index, which passes alone 3 of 3 and is not touched by this diff; looks load-sensitive).🤖 Generated with Claude Code