arXiv AI
Sep 24

Phonemizing User-Generated Text: A Benchmark, Taxonomy, and Compositional Approach

The paper introduces UGTPhon, a grapheme-to-phoneme benchmark for user‑generated text in English, Vietnamese, and Korean, and presents a taxonomy for diagnosing pronunciation errors. It shows that existing G2P models and large language models struggle with canonical‑to‑non‑canonical text, with errors up to 66.8 PER points. A compositional G2P approach that uses exact‑match lookup and staged decoding reduces these errors and performs competitively with larger few‑shot LLMs.

By MinJu Jeon, Younghan Park, Han Sung Park, Jong-Hwan Kim, Dong-Jin Kim, Hoyeon Lee
arXiv Computation and Language
Sep 18

Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Data

The paper introduces a context‑aware neural grapheme‑to‑phoneme (G2P) system for unsegmented languages like Japanese, using a discriminative conditional random field over a word lattice built from dictionaries. It addresses data scarcity by generating over two million sentences with large language models. Experiments show the method surpasses traditional morphological analyzers and neural sequence models, achieving 99.62% target word reading accuracy and very low phoneme error rates on the Joyo‑Kanji‑Yomi benchmark.

By Rui Hu, Zhenpeng Zhan, Xiaolong Lin
Hugging Face Trending Papers
Sep 17

Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Data

The paper introduces a context‑aware neural grapheme‑to‑phoneme (G2P) system for unsegmented languages like Japanese, combining a discriminative conditional random field over a dictionary‑based word lattice with large language model‑generated training data. By generating over two million synthetic sentences, the method addresses data scarcity and achieves superior performance compared to traditional morphological analyzers and neural sequence models. On the Joyo‑Kanji‑Yomi benchmark, it attains 99.62% target word reading accuracy, 0.32% target word phoneme error rate, and 0.14% sentence phoneme error rate.