Skip to content

docs(email-security): body similarity groups by normalization, not by tolerance - #390

Open
maximelb wants to merge 3 commits into
masterfrom
tlsh2-body-similarity-wording
Open

docs(email-security): body similarity groups by normalization, not by tolerance#390
maximelb wants to merge 3 commits into
masterfrom
tlsh2-body-similarity-wording

Conversation

@maximelb

@maximelb maximelb commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Why

The campaigns page's Body similarity section made two claims from single measurements and
they read as guarantees:

  • "86 apart before normalization and 0 apart after it" — real, but one synthetic pair;
  • a warning that a single unremovable per-recipient word "has been measured at 100" — also
    real, and also one pair.

Re-measured across a whole corpus rather than on one pair, the cost of one per-copy word the
normalization cannot identify has a median of 20–30 and a p95 over 100 (worst 201). So the
old text overstates the failure in the typical case and understates the variance, and — the
part that matters operationally — it leaves "raise the threshold" looking like a reasonable
response. It is not: moving 30 to the ceiling of 35 takes that case from roughly half to
roughly two thirds while spending most of the margin against unrelated mail.

Meanwhile the product side of this shipped five more normalizers (all recipients' names, wider
greetings, signature blocks, quoted-reply tails, short mixed letter-and-digit references, and
whole-number collapse), so the list of what the normalization removes was out of date and the
benign-corpus margin moved 39 → 40.

What changed

  • The normalization list now describes what is actually removed.
  • The measured example is now a kit templated over eight recipients in three shapes
    (individually addressed, shared mailbox, thread-hijack) — 37–219 apart raw, 0 apart
    normalized — rather than one pair.
  • The warning admonition is rewritten so the threshold is not read as a tolerance dial, and
    says plainly that raising it is not the lever it looks like.
  • 39 → 40 for the closest unrelated pair, in both places it appears.

No behaviour change here; this is the public wording catching up with the measurement.

Hold

Do not merge yet — this describes normalizers that are not in production. Merge with the
prod release, same as #388 / #389.

… tolerance

The campaigns page quoted one synthetic pair (86 raw / 0 normalized) next to a
warning that a single unremovable word 'has been measured at 100'. Both numbers
are real and both are single draws; measured across the whole corpus the cost of
one unidentified per-copy word is a median of 20-30 with a p95 over 100. Rewrites
the section so the threshold is not read as a tolerance dial, and documents the
normalizers that were added to remove the variance instead.
@maximelb
maximelb marked this pull request as ready for review September 8, 2026 08:18
@maximelb

maximelb commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

/lc-review

lcbill
lcbill previously approved these changes Sep 8, 2026
@maximelb

maximelb commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

HOLD — do not merge yet. Approved and green, but this describes normalizers that are not in production: it depends on go-mailsec#134 and its legion_mailsec re-pin reaching the prod datacenters. Merge with the prod release, alongside #388 and #389.

@maximelb

maximelb commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

/lc-review

lcbill
lcbill previously approved these changes Sep 8, 2026
The benign corpus is no longer 404 messages (the managed pack's own work grew it),
so both pages stop quoting a denominator that will keep drifting. The policy
reference also still quoted the pre-normalizer closest pair of 39 and an example
that is no longer the one the number comes from, and it did not carry the reason
raising the threshold is not the lever it looks like - which is the question that
page is where someone asks.
@maximelb

maximelb commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

/lc-review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants