docs(automod): the rule examples an operator needs, and a test that runs them - #820
Merged
Merged
Conversation
…uns them
AUTOMOD.md said "your regexes" and stopped there. No example, no flavour, and
no mention of the three behaviours that decide whether a pasted pattern works:
* Patterns are Go RE2, compiled with (?i) ALREADY PREPENDED. Nobody needs to
write (?i), and no rule can be case-sensitive.
* Matching is SUBSTRING, so `spam` fires on "a spammy sentence".
* Each pattern is tried THREE WAYS -- raw, normalised, and despaced when the
spacing looks deliberate -- which is why `free\s*robux` also catches
"f r e e r o b u x" and `badword` catches "ssssbadword".
RE2 has no backtracking, so lookahead, lookbehind and backreferences do not
compile. `(.)\1{9,}` is called out by name because it is the obvious way to
write "the same character ten times" and the history checker's repeat detection
is where that job belongs.
ALSO ADDED: the history checker's twelve fields with their defaults, which
appeared nowhere; and the timeout-of-zero trap, which the server refuses but
the documentation never mentioned -- easy to hit through the API, where
omitting a field is easier than in the console.
THE EXAMPLES ARE EXECUTED, NOT TRUSTED. docs_examples_test.go parses the
"Examples that work" table, compiles each pattern, and fires it at the phrase
the table claims it catches -- plus an innocuous sentence, because an example
matching ordinary chat trains an operator to ignore the checker. A documented
regex is pasted into a form and believed, so the cost of a wrong one is not a
typo: it is a rule that silently catches nothing.
Watched fail: a row claiming a phrase it cannot catch, and a renamed heading
that leaves the parser matching nothing. The second is why the test refuses to
pass on fewer than six exercised phrases.
Verified along the way that a worry was unfounded: Normalise collapses doubled
letters ("followers" -> "folowers"), which looked like it would break every
pattern containing one. RuleSet.Check tries the raw message too, so it does not.
Claude-Session: https://claude.ai/code/session_01A8N3W5ct9SZtHK9sCDD9cL
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved documentation and test-validation issues remain.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
This PR expands Automod documentation and adds executable tests for its documented regex examples.
Changes:
- Documents RE2 matching, normalization, despace behavior, and examples.
- Documents history-checker fields, defaults, and timeout semantics.
- Tests that documented patterns compile and match expected phrases.
File summaries
| File | Summary and review notes |
|---|---|
docs/AUTOMOD.md |
Expands operator guidance. Open comments: moderate (1 vote) on inline flag overrides; nit (3 votes) on maxAuthors; moderate (2 votes) on timeout semantics. |
internal/automod/docs_examples_test.go |
Validates documented examples. Open moderate comment (3 votes): empty quoted phrases can silently skip a row. |
Review details
Suppressed comments (1)
docs/AUTOMOD.md:42
- This claim is not true for Go's RE2 syntax: the prepended
(?i)can be overridden by an inline flag such as(?-i:spam), which makes the group case-sensitive. Either reject flag-disabling constructs inRule.Compileif case-insensitivity is meant to be absolute, or document that(?i)is only the default.
You never need to write `(?i)` yourself, and there is no way to make a rule
case-sensitive.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+123
to
+124
| On every platform. The server refuses to save a timeout action carrying no | ||
| duration rather than accepting it and surprising you later: |
| continue | ||
| } | ||
|
|
||
| for _, phrase := range quoted(claims) { |
Comment on lines
+95
to
+97
| The rules checker needs patterns from you; this one ships working defaults and | ||
| is deliberately forgiving. Every field below is what the console writes and what | ||
| the API accepts. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AUTOMOD.mdsaid "your regexes" and stopped there. No example, no flavour, and no mention of the three behaviours that actually decide whether a pasted pattern works.What was missing
(?i)already prepended(?i), and no rule can be made case-sensitivespamfires on "a spammy sentence"free\s*robuxalso catchesf r e e r o b u x;badwordcatchesssssbadwordThat third one is the good part of the design and it was entirely undocumented.
RE2 has no backtracking, so lookahead, lookbehind and backreferences don't compile.
(.)\1{9,}is called out by name — it's the obvious way to write "the same character ten times over", and the history checker's repeat detection is where that job belongs.Also added
The history checker's twelve fields with their defaults, which appeared nowhere. The doc said "rate, repeats, links, mentions, capitals over a window" and named no field, no unit, no default — an operator configuring it had nothing to go on.
The timeout-of-zero trap. The server refuses it with a good message; the documentation never mentioned it. Easy to hit through the API, where omitting a field is easier than in the console.
The examples are executed, not trusted
docs_examples_test.goparses the "Examples that work" table, compiles each pattern, and fires it at the phrase the table claims it catches — plus an innocuous sentence, because an example that matches ordinary chat trains an operator to ignore the checker.A documented regex gets pasted into a form and believed. The cost of a wrong one isn't a typo; it's a rule that silently catches nothing while somebody thinks their chat is filtered.
Watched fail before being trusted:
says "badword" catches "a phrase it cannot possibly catch", and it does nothas no "### Examples that work" section … this test proves nothingThe second is why it refuses to pass on fewer than six exercised phrases.
One scare that turned out to be nothing
Normalisecollapses runs of the same character, so"buy cheap followers"→"buy cheap folowers"and"FREE ROBUX"→"fre robux". That looked like it would break every pattern containing a doubled letter, and I nearly filed it as a defect.It doesn't:
RuleSet.Checktries the raw message as well as the normalised one. I'd probed only the normalised form. The three-form match is exactly what makes both the plain pattern and the evasion-resistant one work at once.https://claude.ai/code/session_01A8N3W5ct9SZtHK9sCDD9cL