arXiv Computation and Language By Wen Zhang

Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection

Read the original on arXiv Computation and Language →

The paper investigates Japanese past‑tense verb inflection in neural morphological generation systems, focusing on a rare irregular subtype where stems end in /e/ and require gemination before the suffix. Despite overall high accuracy (>97%), this <1% subclass accounts for 30–43% of residual errors, amplifying its impact by 34–48 times its frequency. Ablation experiments show that removing this specific subtype yields larger accuracy gains than removing all irregular verbs, highlighting the role of low‑frequency patterns and orthographic processes in error concentration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Sep 4

Contextual Tamil Spelling and Grammar Correction Using Progressively Fine-Tuned Sequence-to-Sequence Transformers

The paper presents an end‑to‑end sequence‑to‑sequence approach for correcting Tamil spelling and grammar errors, leveraging progressively fine‑tuned transformer models (mT5‑small and mBART‑50). Using a synthetic corpus of 657,720 noisy‑clean sentence pairs across ten error categories, the authors introduce a four‑stage training schedule that targets surface noise, contextual grammar, single‑site sandhi, and multi‑site cross‑word sandhi. The best model, mBART‑50 v5, achieves 69.3% exact‑match accuracy on a balanced diagnostic set, with notable gains in sandhi (87.5%) and subject‑verb agreement (43.5%) accuracy, while also revealing a precision‑recall trade‑off for sandhi corrections.

By Karthikeyan A, Jaya Nirmala S, Sangeetha Sivanesan, Indhu R, Pranav Kumar, Bharat Jude Johnson, Vishnu Ram
arXiv Machine Learning
Sep 7

Low-Latency Spell Correction for Japanese Music Search Queries

The paper introduces a compact BART-based sequence‑to‑sequence model for low‑latency spell correction of Japanese music search queries, addressing challenges posed by four writing scripts. It employs a script‑aware synthetic misspelling pipeline that blends keyboard‑layout models, phonetic confusion priors, consonant alternations, and kana case errors, and normalizes mixed‑script titles to a single canonical script to reduce hallucinations. Experiments show the model achieves 41.09% exact‑match accuracy and 11.62% CER, outperforming edit‑distance baselines while keeping inference latency under 4 ms on a single GPU.

By Anshul Garg, Pavni Tandon, Karan Bhukar, Tanmay Khandelwal, Ujjal Kumar Dutta
arXiv Computation and Language
Aug 27

One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography

The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.

By Muge Zhang, Aaron Jencks, Krishna Badikela, Yulia Tsvetkov, Sachin Kumar
arXiv Computation and Language
Sep 1

Quantifying and Mitigating Korean Jamo-Level Typographical Vulnerabilities in Large Language Models

The paper investigates a unique typographical vulnerability in Korean language models arising from errors at the jamo (sub-character) level, which can produce valid but altered syllables or expose raw jamo, thereby disrupting tokenization and bypassing standard error correction. By applying five jamo-level perturbations to the KMMLU benchmark, the authors show that model accuracy degrades steadily with perturbation intensity and that larger models do not gain robustness. They further demonstrate that corrupted inputs shift internal representations in a detectable way, enabling a linear probe to identify unseen typos and motivate a Typo-Aware Chain-of-Thought (TACoT) strategy that selectively triggers chain-of-thought inference only when a typo is detected, recovering much of the accuracy benefit at lower cost.

By Seojin Lee, Hwanhee Lee