Don't apply self-referential replacements twice - #179
binggao1230 wants to merge 1 commit into
Conversation
Custom replacements run in two passes: once before unidecode and once in the finalize step (so rules can re-fire on tokens that reappear after conversion, per issue un33k#119). For a self-referential rule (the replacement contains its own search value, e.g. ['a', 'aa']) the finalize pass re-fires on the first pass's output and grows the slug each run: slugify('a', replacements=[['a', 'aa']]) returned 'aaaa' instead of 'aa'. Skip only those self-referential rules in the finalize pass. The intentional re-scan is preserved for every other rule, and all existing replacement behavior is unchanged.
|
Closing in favor of #182 which supersedes this — it covers both passes (not just pass-2), handles indirect cycles (where |
|
This is Dojo, posting a maintainer-authorized follow-up linking this PR to #191. The proposed automatic skipping of self-referential rules remains declined. Explicit pre/post replacement stages are available instead; reusable rules retain the intentional two-pass default. No universal idempotence guarantee is introduced. The default algorithm remains legacy; improved output rules are opt-in. This note does not announce a published release, and no individual PR is being merged by this follow-up. Thank you for the contribution and discussion. 🚀 Generated with Dojo ⛩️ |
The problem
Custom
replacementsare applied in two passes: once before unidecode (line ~109) and once in the finalize step (line ~186). As discussed in #119, the second pass is intentional so that rules can re-fire on tokens that reappear after conversion, especially for non-English text.That second pass corrupts a self-referential rule, where the replacement value contains its own search key. The finalize pass re-fires on the first pass's output and the slug grows on every run:
The fix
Keep the intentional finalize re-scan, but skip the rules where re-application is provably self-corrupting (
old in new) — those were already applied in the first pass and cannot be safely re-applied:Every non-self-referential rule still gets the second pass exactly as before, so the re-scan behavior from #119 is untouched. The README examples (
[['|', 'or'], ['%', 'percent']]→10-or-20-percent, the German umlaut customs, the&workaround) all still produce identical output.Tests
Added
test_replacements_not_applied_twicetoTestSlugify; it fails before the change ('aaaa' != 'aa') and passes after. Full suite: 83 tests pass;pycodestyle(repo flags) clean;mypy slugifyclean. Added a CHANGELOG entry under Unreleased.Since the two-pass behavior is deliberate, I kept the change as narrow as possible — happy to adjust if you'd prefer a different approach (e.g. an explicit pre/post split).
I used an AI assistant to help investigate this under my direction, and I have reviewed and verified the change myself.