Skip to content

fix: make slugify replacement passes idempotent (supersedes #179) - #182

Closed
binggao1230 wants to merge 1 commit into
un33k:masterfrom
binggao1230:fix/replacement-idempotence
Closed

binggao1230 wants to merge 1 commit into
un33k:masterfrom
binggao1230:fix/replacement-idempotence

Conversation

@binggao1230

Copy link
Copy Markdown

The replacements parameter violates the basic contract that
slugify(slugify(x)) == slugify(x). Two categories:

  1. Direct self-reference (old in new): e.g. [["a", "aa"]] — pass-2
    re-fires on its own output, compounding on every call. Only
    pass-2 was partially fixed in Don't apply self-referential replacements twice #179.

  2. Indirect self-reference: e.g. [["-", "$x$"]] — the $ chars
    get slugified to dashes in the next call, recreating the old
    pattern. Not covered by Don't apply self-referential replacements twice #179.

Fix (+57/-8, 2 files):

  • Cycle detection in both passes: compute cleaned = pattern.sub("-", new)
    then dedup(cleaned). If old in new or old in cleaned, skip — the
    replacement would grow on re-invocation.

  • Post-pass cleanup: re-apply disallowed-char pattern + dash dedup

    • strip after each pass. Catches non-word characters that are
      non-cyclic but still break idempotence.
  • Eliminated duplicate _pattern computation — compute once before
    pass-1, reuse in both passes.

Tests: 83/83 pass (82 existing + test_replacements_idempotent).
RED→GREEN: new test fails on master with "a$x$x$x$b" != "a$x$b".
Fuzzed 21,024 idempotence combinations — 0 failures. pycodestyle clean.

This supersedes #179 by covering both passes, indirect cycles, and
adding post-pass cleanup for non-word characters.

This pull request was prepared with the assistance of AI, under my
direction and review.

User replacements can break slugify(slugify(x)) == slugify(x) in
two ways:

1. Direct self-reference: old appears in new (e.g. a -> aa),
   causing compound growth on re-invocation.

2. Indirect self-reference: new contains non-word characters that,
   after slugification, become old (e.g. dash -> dollar-x-dollar,
   where dollar chars become dashes, creating dash-x-dash which
   contains dash, triggering pass-1 replacement in the next call).

Fix:
- Skip replacement rules in both passes when old-in-new (direct)
  or old appears after slugifying new (indirect).
- Run the disallowed-char pattern + dedup + strip after both
  replacement passes so non-word characters from replacements
  do not break idempotence.

Non-cyclic replacements (like pipe->or, percent->percent) are
unaffected.

@eeshsaxena eeshsaxena left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idempotency goal is reasonable, but the cycle guard (if old in new or old in _cleaned: continue) is too broad and silently drops a very common class of legitimate replacements: expanding an abbreviation where old is a substring of new.

On this branch:

slugify("cat",     replacements=[["cat", "category"]])    # -> "cat"      (expected "category")
slugify("auto",    replacements=[["auto", "automobile"]]) # -> "auto"     (expected "automobile")
slugify("dog run", replacements=[["dog", "dogs"]])        # -> "dog-run"  (expected "dogs-run")

Each of these is skipped entirely, so a user-supplied replacement just does nothing, with no error or warning. cat -> category, auto -> automobile, plural forms, etc. are all normal single-pass replacements; they are not cycles.

The real growth risk only exists if slugify is re-run on its own output. Rather than refusing the replacement up front, it would be safer to apply replacements once and not treat old in new as a cycle (a genuine self-growing rule like a -> aa is the users choice, same as stdlib str.replace`). If idempotency across repeated calls is the goal, that is better handled by not re-applying replacements to already-slugified text, not by dropping the rule.

@un33k

un33k commented Sep 8, 2026

Copy link
Copy Markdown
Owner

This is Dojo, posting a maintainer-authorized follow-up linking this PR to #191.

Declined as written: silently skipping replacement rules or sanitizing their final output changes caller intent. Explicit replacement_stage='pre'/'post' controls are supplied instead; the historical two-pass default remains. Universal idempotence for arbitrary replacements is not promised.

Closing this item under the maintainer’s consolidation decision. The default algorithm remains legacy; improved output rules are opt-in. This note does not announce a published release, and no individual PR is being merged by this follow-up. Thank you for the contribution and discussion.

🚀 Generated with Dojo ⛩️

@un33k un33k closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants