fix(import): an import that changes a stored message's content puts its duplicate flag right

[mbeisser1]

Bug Description

Expected: An import that changes a stored message's content (a later edit's text, an attachment added, or a stored attachment given its file) leaves the message's duplicate flag matching its new content, whatever the import's dedupe setting.

Actual: An append with dedupe off changed the content and left the flag. Message M hidden behind N, another source's copy of "see you at six", took the text "see you at seven" and stayed hidden behind N, so a search for "seven" found nothing. Copies hidden behind M stayed hidden though their text no longer matched.

Root Cause

Only dedupe::dedupe_cross_source sets or clears duplicate_of, and run_import_path in crates/server/server/src/imports_api/mod.rs runs it only when the Import Run has dedupe on. Since #1801 and #1966, promote_later_edits changes a stored message's text and clears its content key, and promote_attachments changes what the key hashes, but nothing re-evaluated the flag when dedupe was off.

Fix Description

Option 1 of the decision on #1805: the import re-runs the dedupe for the changed messages, whatever its setting.

• promote_attachments (db/staging.rs) now returns the stored messages whose attachment rows took their file (RETURNING message_id), and stored_messages_with_new_content names the messages in _promote_edit_map and those that gained an attachment row.
• Promote::run (imports_api/promote.rs) passes those ids to the new dedupe::dedupe_changed_messages, inside the import's transaction, as its last phase. With no changed message it runs nothing.
• dedupe_changed_messages computes the changed messages' content keys again, runs the exact and near-time passes over the messages with a content key, and writes the flags of the messages tied to a changed one: those it hid or was hidden behind, before or now, followed to the end, so no hidden message is left pointing at one that is hidden. A message an import with dedupe off added has no content key, so it is neither hidden nor a winner. Every other flag stays as it is.
• dedupe_cross_source shares the pieces: the exact pass is split into compute and apply, and the near-time pass reads the exact pass's losers from memory instead of duplicate_of IS NULL (the same rows, since the pass clears every flag first). NEAR_WINDOW_SECS names the 2-second window both use.
• docs/architecture/contacts-identities-and-messages.md states the rule in place of the "still leaves" note. A CHANGELOG entry is under 0.11.0, Fixes, Importing.

Assumptions, stated rather than asked:

• The pass runs with dedupe on too, before the import's own full pass, so the import command with --skip-dedupe (which fills content keys but runs no pass) is covered as well. With dedupe on it is redundant work, only when stored content changed.
• A deletion mark change is not a content change here: the content key does not hash it.
• The issue's acceptance criteria say the CHANGELOG entry goes under 0.10.0; it is under the in-development 0.11.0 heading, as the brief said.

How to Reproduce (Before Fix)

1. Import a message "see you at six" from source sms, then the same message from source imessage, both with dedupe on: the imessage copy is hidden behind the sms one.
2. Append, with dedupe off, a later backup in which the imessage copy was edited to "see you at seven".
3. GET /v1/messages?q=seven finds nothing, and the imessage copy is still hidden.

How to Verify (After Fix)

1. cargo test -p message-crate-server changed_content_dedupe runs the four new tests in crates/server/server/src/imports_api/tests/changed_content_dedupe.rs, through the HTTP import routes.
2. an_edit_with_dedupe_off_shows_a_message_hidden_behind_a_copy_of_its_old_text is the issue's scenario: M is shown, a search for "seven" finds it, and a new message the same import brings stays shown beside the copy it duplicates.
3. an_edit_with_dedupe_off_evaluates_the_copies_around_the_message_again: the copy of M's old text hidden behind M is shown again, and a third source's copy of the new text is hidden behind M. an_attachment_with_dedupe_off_keeps_a_copy_that_still_matches_hidden: a copy that still matches M stays hidden, and M's content key hashes its new file. an_append_with_dedupe_off_that_changes_nothing_stored_runs_no_dedupe: no change, no pass.

The first three fail without the fix. The fourth and the "new message stays shown" assertion pass without it by their nature: they guard the scope of the fix against an account-wide pass.

Impact Assessment

• Severity: Medium. A message disappears from lists and search, and nothing says why.
• Users affected: Subset: accounts that hide duplicates across sources and later append, with dedupe off, a backup in which a stored message was edited or gained an attachment.
• Duration: Since fix(server): an append from a later backup takes a stored message's later edit #1801 made an append take a stored message's later edit; an added attachment has changed the content key since appends could add one.

Regression Risk

Every import that changes a stored message's content now loads the account's keyed messages once for the two passes, which costs about what the full pass costs less the key hashing. Imports that change nothing stored are unaffected. The refactor of dedupe_cross_source keeps its behaviour: the whole server suite passes, including the generated-database invariant test.

Checklist

• Root cause identified and documented above
• Fix addresses the root cause (not just symptoms)
• Added test to prevent regression
• Existing tests pass locally (cargo test -p message-crate-server: 1326 passed, 1 ignored; ./scripts/check-pr.sh passed)
• Tested the specific reproduction steps

Related Issues

Closes #1805

Related: #1801, #1757.

🤖 Generated with Claude Code

[mbeisser1]

Review of a9e1394 (the PR with main merged in). Correctness 2, Spec 3, Standards 5. Findings with no line in the diff follow in a top-level comment. Each is answered in its thread.

[mbeisser1]

Re-review of the fix commit 028aec2. Standards 5, Correctness 1. Findings with no line in the diff follow in a top-level comment. Each is answered in its thread.

[mbeisser1]

Review summary. Reviewed head a9e1394 (the PR with main merged in), fixes 028aec2 and 2f4e61c, base merged in again as c4a5d9e (no conflicts), CI green on c4a5d9e.

• Correctness: 2 findings (one bug: a keyless message was keyed and hidden), 2 fixed. Re-review: 1, fixed.
• Spec: 3 findings, 2 fixed, 1 recorded on An append with dedupe off leaves a changed message's duplicate flag #1805 (a guard test that passes without the fix, kept on purpose).
• Standards: 5 findings, 5 fixed. Re-review: 5, 5 fixed.
• Deferred: none. No commits for CI failures. Both merges of the base were conflict-free.
• No user threads.