Avoid trailing post-replacement delimiters in modern hard cuts - #200
Conversation
|
Thank you, @Eric3-jp — verified locally and this is a solid fix. A modern hard cut through repeated post-replacement delimiters was leaving a trailing delimiter ( Local verification: full suite 125 passed / 117 subtests, the 2,688-case legacy differential passes (legacy output unchanged), the #193 fitting fast path still holds, mypy clean, and repo pycodestyle/flake8 pass. Confined to 🚀 Generated with Dojo ⛩️ |
Bump to 9.1.1 for the modern truncation fix (曾楚笑, #200): a hard cut through repeated post-replacement delimiters no longer emits a trailing separator. Legacy output and public smart_truncate are unchanged. Update check_dist.py version assertions to 9.1.1. 🚀 Generated with [Dojo](https://heydojo.ai) ⛩️
With
algorithm='modern', a hard cut through repeated delimiters introduced by a post replacement can leave a trailing delimiter:Keep a run of delimiters with the next nonempty token, and only emit it if at least one character from that token fits. This also handles multi-character separators, leading delimiter runs, and the hard-cut fallback from word-boundary mode. A delimiter that occurs in the word itself is preserved.
This complements #193, which preserves post-replacement output when the whole slug already fits. That fast path, whole-word selection, the default/explicit legacy algorithm, and public
smart_truncateare unchanged.Validation on Windows with CPython 3.14.0:
python -m pytest --color=no -q: 125 passed, 117 subtests passed, including the unchanged legacy suite, the 2,688-case legacy differential, and fitting-post-replacement coverage.git diff --checkpasses.The complete supported-interpreter/tox matrix has not been run locally.
OpenAI Codex assisted with the investigation, patch, tests, and this description. The checks listed above were executed locally.