docs(email-security): body similarity groups by normalization, not by tolerance - #390
Open
maximelb wants to merge 3 commits into
Open
docs(email-security): body similarity groups by normalization, not by tolerance#390maximelb wants to merge 3 commits into
maximelb wants to merge 3 commits into
Conversation
… tolerance The campaigns page quoted one synthetic pair (86 raw / 0 normalized) next to a warning that a single unremovable word 'has been measured at 100'. Both numbers are real and both are single draws; measured across the whole corpus the cost of one unidentified per-copy word is a median of 20-30 with a p95 over 100. Rewrites the section so the threshold is not read as a tolerance dial, and documents the normalizers that were added to remove the variance instead.
maximelb
marked this pull request as ready for review
September 8, 2026 08:18
Contributor
Author
|
/lc-review |
lcbill
previously approved these changes
Sep 8, 2026
Contributor
Author
Contributor
Author
|
/lc-review |
lcbill
previously approved these changes
Sep 8, 2026
The benign corpus is no longer 404 messages (the managed pack's own work grew it), so both pages stop quoting a denominator that will keep drifting. The policy reference also still quoted the pre-normalizer closest pair of 39 and an example that is no longer the one the number comes from, and it did not carry the reason raising the threshold is not the lever it looks like - which is the question that page is where someone asks.
Contributor
Author
|
/lc-review |
lcbill
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The campaigns page's Body similarity section made two claims from single measurements and
they read as guarantees:
real, and also one pair.
Re-measured across a whole corpus rather than on one pair, the cost of one per-copy word the
normalization cannot identify has a median of 20–30 and a p95 over 100 (worst 201). So the
old text overstates the failure in the typical case and understates the variance, and — the
part that matters operationally — it leaves "raise the threshold" looking like a reasonable
response. It is not: moving 30 to the ceiling of 35 takes that case from roughly half to
roughly two thirds while spending most of the margin against unrelated mail.
Meanwhile the product side of this shipped five more normalizers (all recipients' names, wider
greetings, signature blocks, quoted-reply tails, short mixed letter-and-digit references, and
whole-number collapse), so the list of what the normalization removes was out of date and the
benign-corpus margin moved 39 → 40.
What changed
(individually addressed, shared mailbox, thread-hijack) — 37–219 apart raw, 0 apart
normalized — rather than one pair.
says plainly that raising it is not the lever it looks like.
No behaviour change here; this is the public wording catching up with the measurement.
Hold
Do not merge yet — this describes normalizers that are not in production. Merge with the
prod release, same as #388 / #389.