Default to anyascii decoder

[ndrezn]

Default to a non-GPL encoder. This encoder works for most cases, but it could be backwards incompatible.

I recommend a major release as a result if this change is accepted.

See: #128 , #162, etc for more related to this issue. Providing a non-copyleft licensed default package prevents unintended misinterpretation of the package license and putting folks using this package into murky legal water.

[Arian-Ott]

Hi,

thanks for raising this and a happy new year!

I'm aware of the legal implications of GPL licenses and I'm a strong supporter of licensing software correctly.

@ndrezn, do you know if there are significant differences between anyascii and unidecode?

While I couldn't see any differences in a random German sentence I came up with, I cannot verify if the same applies to other languages.

Python 3.14.0 (tags/v3.14.0:ebf955d, Oct  7 2025, 10:15:03) [MSC v.1944 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Ctrl click to launch VS Code Native REPL
>>> from anyascii import anyascii
>>> from unidecode import unidecode 
>>> test = "Baden-Württemberg ist ein Bundesland."  
>>> anyascii(test)
'Baden-Wurttemberg ist ein Bundesland.'
>>> unidecode(test)
'Baden-Wurttemberg ist ein Bundesland.'
>>>

Changing the default package on the fly could cause unwanted side effects we may not be aware of in other projects using slugify.

@un33k what do you think?

[mrezzamoradi]

Examples where unidecode and anyascii differ:

Input:     β
unidecode: b
anyascii:  v

Input:     中文
unidecode: Zhong Wen 
anyascii:  ZhongWen

Input:     你好
unidecode: Ni Hao 
anyascii:  NiHao

Input:     日本語
unidecode: Ri Ben Yu 
anyascii:  RiBenYu

Input:     ♥
unidecode: hearts
anyascii:  :hearts:

Input:     ★
unidecode: 
anyascii:  *

Input:     ©
unidecode: (c)
anyascii:  (C)

Input:     Æ
unidecode: AE
anyascii:  Ae

Input:     مرحبا
unidecode: mrHb
anyascii:  mrhb

Input:     한글
unidecode: hangeul
anyascii:  HanGeul

Here's the code:

from unidecode import unidecode
from anyascii import anyascii

test_cases = [
    # Common accented characters
    'café', 'naïve', 'résumé',
    # German umlauts
    'Müller', 'Größe',
    # Scandinavian
    'Øresund', 'Åse',
    # Greek
    'α', 'β', 'γ', 'Ω',
    # Cyrillic
    'Привет', 'Москва',
    # Chinese
    '中文', '你好',
    # Japanese
    '日本語', 'カタカナ',
    # Emoji and symbols
    '♥', '★', '©', '€',
    # Special characters
    'ß', 'Æ', 'œ',
    # Vietnamese
    'Tiếng Việt',
    # Arabic
    'مرحبا',
    # Korean
    '한글',
]

print("Examples where unidecode and anyascii differ:\n")

for text in test_cases:
    uni_result = unidecode(text)
    any_result = anyascii(text)

    if uni_result != any_result:
        print(f"Input:     {text}")
        print(f"unidecode: {uni_result}")
        print(f"anyascii:  {any_result}")
        print()

[Arian-Ott]

Thanks for sharing . In my opinion, this would be a high-impact breaking change that could lead to significant incompatibilities across the entire ecosystem. If anyascii was 100% compatible, that would be less of a problem to transition to.

python-slugify is core infrastructure: with over 113k public projects and more than 1,000 public packages depending on it (not counting the vast number of private corporate deployments), changing the backend would be too disruptive. Even subtle differences in how slugs are generated could break URLs, file systems, and database lookups for millions of users.

As I also maintain python-slugify downstream in Debian (https://tracker.debian.org/pkg/python-slugify), I’ve checked the local impact. A change would directly affect critical packages like cookiecutter, lektor, and security tools like opensnitch:

arian@debian:~$ apt-cache rdepends python3-slugify
python3-slugify
Reverse Depends:
  python3-blinkpy
  staticsite
  slugify
  python3-pyaarlo
  python3-logi-circle
  python3-agate
  python3-opensnitch-ui
  lektor
  python3-cookiecutter
arian@debian:~$

Furthermore, anyascii relies on precompiled binaries. This is a blocker for many Linux distributions (like Debian), which require everything to be built from source for security and transparency reasons.

Given these points, I believe prioritizing stability and compatibility is the right path forward.

https://qa.debian.org/popcon.php?package=python-slugify

[Arian-Ott]

What could work is if we find a second backend not relying on compiled files under a different license. That way users can choose which backend they want.

[mrezzamoradi]

I agree it's too much of a breaking change even for dumping it as a major version, unless we make it installable with a different backend as @Arian-Ott suggested

[un33k]

Hey folks, I am making some changes to slugify which might render upcoming PRs redundant.

For these changes, I am using Claude Code with my own ehAye Engine.

Check it out ... https://ehaye.io

Cheer,
Val

[ndrezn]

Thanks for the feedback @mrezzamoradi @Arian-Ott ! Useful examples to understand the impact; of the existing tests only two failed with the change -- if nothing else it seems like those examples would be useful to add as test cases to this project.

The second backend idea is a good one; the problem is that the existing dependencies must be installed because of how Python setuptools; it's not possible to specify installing python-slugify without a GPL encoder being installed even if it is used (i.e. retain backwards compatibility), which makes a GPL package a mandatory dependency 😞 . But maybe I'm missing something or you have a different approach in mind.

[Arian-Ott]

The second backend idea is a good one; the problem is that the existing dependencies must be installed because of how Python setuptools; it's not possible to specify installing python-slugify without a GPL encoder being installed even if it is used (i.e. retain backwards compatibility), which makes a GPL package a mandatory dependency 😞 . But maybe I'm missing something or you have a different approach in mind.

That's not how setuptools work.

python-slugify/setup.py

Lines 14 to 15 in 7b6d5d9

	install_requires = ['text-unidecode>=1.3']
	extras_requires = {'unidecode': ['Unidecode>=1.1.1']}

If a user installs slugify using pip install slugify only the base dependency gets installed.
If a user requires the unidecode backend, they can explicitly specify that in the pip install using the pip install python-slugify[unidecode] extension.

In the last scenario both text-unidecode (Artistic or GPL2 at user's choice) AND unidecode (GPL3) gets installed in which case slugify would become GPL3.

I discussed this with an experienced Debian developer and we both came to said conclusion.

Furthermore, introducing a third backend would require some development time which is in no relation to the actual impact. Most users install slugify using pip install python-slugify which by default uses text-unidecode.

[Arian-Ott]

Even if we were to agree on implementing a third backend (which I believe is unnecessary), the overhead is significant: you would have to implement tests, verify licensing, update the README, and document the changes.

Furthermore, from a distribution perspective, I would potentially have to package this new backend for Debian, which involves extensive code reviews and the NEW queue process. Given that perhaps only a handful of people would actually use a third backend, the effort is simply not proportional to the impact. The current system with extras_require already solves the licensing concern for those who need to avoid GPL.

Hope that helps

[un33k]

This is Dojo, posting a maintainer-authorized follow-up linking this PR to #191.

AnyASCII is available as an explicitly selected backend. Switching the automatic default to AnyASCII is declined to preserve existing output; its installation alone does not change auto selection.

The default algorithm remains legacy; improved output rules are opt-in. This note does not announce a published release, and no individual PR is being merged by this follow-up. Thank you for the contribution and discussion.

🚀 Generated with Dojo ⛩️