Skip to content

Make the Kirat Rai ligature states ligature-transparent - #95

Open
youdie006 wants to merge 1 commit into
unicode-rs:masterfrom
youdie006:kirat-rai-ligature-transparent
Open

Make the Kirat Rai ligature states ligature-transparent#95
youdie006 wants to merge 1 commit into
unicode-rs:masterfrom
youdie006:kirat-rai-ligature-transparent

Conversation

@youdie006

@youdie006 youdie006 commented Sep 11, 2026

Copy link
Copy Markdown

The crate measures the same Kirat Rai ligature two ways:

"\u{16D68}".width()                  // 1  precomposed
"\u{16D63}\u{16D67}".width()         // 1  decomposed
"\u{16D63}\u{200D}\u{16D67}".width() // 2  decomposed + ZWJ

KIRAT_RAI_VOWEL_SIGN_E and _AI (src/gen/width_info.rs:39-40) are the only two of eight script-ligature states carrying none of the ligature-transparency bits, so is_ligature_transparent is false and src/lookup.rs:149-155 never skips the ZWJ. scripts/unicode.py:214-215 states the rule they break: "5th from top: whether this is unaffected by ligature-transparent (if set, should also set 3rd and 4th)". #53 established the convention for the first six states; #74 added Kirat Rai without it.

The new constants use the same 0b0011_1100_ prefix as LISU_TONE_LETTER_MYA_NA_JEU and KHMER_COENG_ELIGIBLE_LETTER. The edit is in scripts/unicode.py; src/gen/width_info.rs is regenerated output.

Scope and verification

Differential over every Unicode scalar value as the inserted character, six <base><mid><closer> sequences, pristine vs fixed, both width and width_cjk:

sequence cells changed transitions
Kirat Rai 16D63..16D67 528 all 2 -> 1
Kirat Rai 16D63..16D68 528 all 2 -> 1
Hebrew / Lisu / Khmer / emoji ZWJ 0
row result
new test on master FAILED, "𖵣\u{200d}𖵧" has the wrong width, left 2 right 1
with the fix ok, 37 passed
revert FAILED
transparent bit without the ZWJ bit FAILED
fix _E only FAILED

CI rows run locally: cargo test, --no-default-features, cargo fmt --check, clippy both feature sets under -D warnings, cargo doc under -D warnings, cargo +1.66.0 check --lib --features core,std — all exit 0. The regen gate (rm tests/emoji-test.txt && cd scripts && python3 unicode.py, then git update-index --refresh && git diff-index --quiet HEAD --) exits 0 on the committed tree, and all five src/gen/*.rs md5s are unchanged by that regeneration.

One thing I did not check: the Unicode core spec text on whether Kirat Rai vowel-sign ligatures must be ligature-transparent. I verified only that this crate is inconsistent with itself, with its generator's stated bit contract, and with its own docs. If the intended answer is the other direction, src/lib.rs:80 is what needs changing and I will send that instead.


Disclosure: I used Claude (an AI assistant) while preparing this change. Every result above I ran and verified myself.

KIRAT_RAI_VOWEL_SIGN_E and _AI were the only two script ligature
states without the ligature-transparent and ZWJ bits, so inserting a
ZWJ or a default-ignorable mark into a Kirat Rai ligature widened it
from 1 to 2 while every other script ligature was unaffected.
@Jules-Bertholet

Copy link
Copy Markdown
Contributor

Thanks for the report!

I am inclined to say we should resolve this by changing the documentation. The Kirat Rai rules are intended only to ensure canonically equivalent strings have the same width, nothing beyond that. Unfortunately there is no Unicode spec for this, and no Kirat Rai monospace fonts.

(As an aside, the excessively verbose Claude output is quite grating to read in PR descriptions, and wastes everyone's time. Please try to cut it down in the future)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants