Skip to content

Avoid trailing post-replacement delimiters in modern hard cuts - #200

Merged
un33k merged 1 commit into
un33k:masterfrom
Eric3-jp:fix/modern-truncate-trailing-delimiters
Sep 22, 2026
Merged

un33k merged 1 commit into
un33k:masterfrom
Eric3-jp:fix/modern-truncate-trailing-delimiters

Conversation

@Eric3-jp

Copy link
Copy Markdown
Contributor

With algorithm='modern', a hard cut through repeated delimiters introduced by a post replacement can leave a trailing delimiter:

slugify(
    'value',
    algorithm='modern',
    replacements=[('value', 'one--two')],
    replacement_stage='post',
    max_length=5,
)
# Before: 'one-'
# After:  'one'

Keep a run of delimiters with the next nonempty token, and only emit it if at least one character from that token fits. This also handles multi-character separators, leading delimiter runs, and the hard-cut fallback from word-boundary mode. A delimiter that occurs in the word itself is preserved.

This complements #193, which preserves post-replacement output when the whole slug already fits. That fast path, whole-word selection, the default/explicit legacy algorithm, and public smart_truncate are unchanged.

Validation on Windows with CPython 3.14.0:

  • The new regressions fail on the original implementation (6 failing subcases and 1 failing fallback test).
  • python -m pytest --color=no -q: 125 passed, 117 subtests passed, including the unchanged legacy suite, the 2,688-case legacy differential, and fitting-post-replacement coverage.
  • Strict Mypy 1.20.2 and repository-configured pycodestyle/flake8 commands pass.
  • git diff --check passes.

The complete supported-interpreter/tox matrix has not been run locally.

OpenAI Codex assisted with the investigation, patch, tests, and this description. The checks listed above were executed locally.

@un33k

un33k commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Thank you, @Eric3-jp — verified locally and this is a solid fix. A modern hard cut through repeated post-replacement delimiters was leaving a trailing delimiter (one-); keeping the delimiter run with the next nonempty token and only emitting it when a character fits correctly yields one. I confirmed it handles multi-character separators, leading delimiter runs, and the word-boundary fallback, while a delimiter that occurs inside a word is preserved.

Local verification: full suite 125 passed / 117 subtests, the 2,688-case legacy differential passes (legacy output unchanged), the #193 fitting fast path still holds, mypy clean, and repo pycodestyle/flake8 pass. Confined to _modern_truncate; the legacy algorithm and public smart_truncate are untouched. Merging now.

🚀 Generated with Dojo ⛩️

@un33k
un33k merged commit 8f9a550 into un33k:master Sep 22, 2026
un33k added a commit that referenced this pull request Sep 22, 2026
Bump to 9.1.1 for the modern truncation fix (曾楚笑, #200): a hard cut
through repeated post-replacement delimiters no longer emits a trailing
separator. Legacy output and public smart_truncate are unchanged. Update
check_dist.py version assertions to 9.1.1.

🚀 Generated with [Dojo](https://heydojo.ai) ⛩️
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants