Skip to content

First learned Urdu G2P + weak-supervision diacritization - #1

Open
ronaldtse wants to merge 2 commits into
mainfrom
feat/urdu-g2p-and-publish
Open

First learned Urdu G2P + weak-supervision diacritization#1
ronaldtse wants to merge 2 commits into
mainfrom
feat/urdu-g2p-and-publish

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Summary

  • First learned Urdu G2P baseline: ByT5 on the 635K-entry grapheme-IPA dictionary — 14.77% CER, 4.1x over the epitran rule baseline (60.0%).
  • Weak-supervision diacritization: IPA-to-haraqat converter producing 597K noisy labels trains a 3.74% CER diacritizer with no gold corpus.
  • docs/RESULTS.md + Urdu paper + epitran baseline artifacts.

Test plan

  • docs/RESULTS.md + paper compile
  • epitran baseline reproduced (scripts/epitran_baseline.py)
  • data-diacrit/ gitignored (rebuildable via scripts/)

ByT5 on the 635K-entry grapheme-IPA dictionary (14.77% CER, 4.1x over
the epitran rule baseline of 60.0%), plus an IPA-to-haraqat converter
producing 597K noisy labels for diacritization (3.74% CER).
First learned Urdu G2P baseline and weak-supervision diacritization;
epitran baseline artifacts included.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant