The study investigates whether morphological correctness influences tokenization effectiveness in Tagalog, a language with infix morphology that disrupts contiguous roots. Researchers developed MorphAustro‑Tok, a tokenizer that preserves the root as a contiguous unit and separates infixes, and compared it to a standard BPE baseline and a scrambled‑morpheme control that randomizes infix placement. Across multiple runs, MorphAustro‑Tok consistently outperformed the baseline, but the scrambled control showed no significant difference, indicating that morphological correctness does not contribute to the observed gains at this scale.
By Abdul Rehman Antall, Baidya Nath Saha
The paper presents an end‑to‑end sequence‑to‑sequence approach for correcting Tamil spelling and grammar errors, leveraging progressively fine‑tuned transformer models (mT5‑small and mBART‑50). Using a synthetic corpus of 657,720 noisy‑clean sentence pairs across ten error categories, the authors introduce a four‑stage training schedule that targets surface noise, contextual grammar, single‑site sandhi, and multi‑site cross‑word sandhi. The best model, mBART‑50 v5, achieves 69.3% exact‑match accuracy on a balanced diagnostic set, with notable gains in sandhi (87.5%) and subject‑verb agreement (43.5%) accuracy, while also revealing a precision‑recall trade‑off for sandhi corrections.
By Karthikeyan A, Jaya Nirmala S, Sangeetha Sivanesan, Indhu R, Pranav Kumar, Bharat Jude Johnson, Vishnu Ram
arXiv:2608.09289v2 Announce Type: replace
Abstract: Second language writing research distinguishes grammatical accuracy from native-like idiomaticity, yet automated writing evaluation often conflates...
By Steve Woollaston, Brendan Flanagan, Hiroaki Ogata
MoVoC introduces a morphology-aware subword construction method for Ge'ez script languages, integrating supervised morphological analysis with Byte Pair Encoding to preserve morphological boundaries while retaining lexical meaning. The resulting MoVoC-Tok tokenizer outperforms baselines in MorphScore and morpheme boundary precision across several low-resource, morphologically complex languages, notably achieving significant gains on Tigre through cross-lingual application. The authors also release morpheme-annotated datasets and a morpheme-aware vocabulary for public use.
By Hailay Kidu Teklehaymanot, Dren Fazlija, Wolfgang Nejdl
arXiv:2606. 15144v1 Announce Type: cross Abstract: Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation.
By Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan, Richell Isaiah Flores, Ivan Yuri De Leon, Lance Calvin Gamboa
The paper introduces a compact BART-based sequence‑to‑sequence model for low‑latency spell correction of Japanese music search queries, addressing challenges posed by four writing scripts. It employs a script‑aware synthetic misspelling pipeline that blends keyboard‑layout models, phonetic confusion priors, consonant alternations, and kana case errors, and normalizes mixed‑script titles to a single canonical script to reduce hallucinations. Experiments show the model achieves 41.09% exact‑match accuracy and 11.62% CER, outperforming edit‑distance baselines while keeping inference latency under 4 ms on a single GPU.
By Anshul Garg, Pavni Tandon, Karan Bhukar, Tanmay Khandelwal, Ujjal Kumar Dutta
arXiv:2610.19041v1 Announce Type: new
Abstract: Understanding the dynamics of language acquisition in Large Language Models (LLMs) is crucial for uncovering how statistical representations evolve int...
By Forough Shirin Abkenar, Haoshuo Fu, Liang Feng, Ning Yan, Masood S. Mortazavi
Large language models show strong cross‑lingual abilities, but previous studies have examined them only at isolated training points. This work pretrains a 1.7B multilingual model on nine languages, recording checkpoints at fine granularity to track translation development. Using word‑level translation and a new dataset, the authors find that early learning is dominated by token‑level copying and surface similarities, while later stages involve more general translation mechanisms and refined copying.
By Felicia K\"orner, Maria Matveev, Florian Eichin, Gitta Kutyniok, Barbara Plank, Michael A. Hedderich
The study evaluates phonologically informed tokenization for German speech recognition using a wav2vec 2.0 backbone fine‑tuned with CTC. Three tokenizer families—pretrained multilingual characters, data‑driven BPE, and phonologically informed units from syllabification and grapheme‑to‑phoneme conversion—are compared across 40 fine‑tunes on three German test sets (in‑domain read, dialectal spontaneous, and standard spontaneous speech). Results show that phonologically informed tokenizers match BPE and character baselines in‑domain, but under domain shift their performance depends on vocabulary size rather than linguistic variation, with syllable‑aware tokenization benefiting dialectal and spontaneous speech at small vocabularies; a phoneme‑level confusion analysis indicates that the acoustic encoder dominates error patterns.
By Christopher Witzl, Tobias Bocklet, Korbinian Riedhammer
The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.
By Muge Zhang, Aaron Jencks, Krishna Badikela, Yulia Tsvetkov, Sachin Kumar
The paper investigates a unique typographical vulnerability in Korean language models arising from errors at the jamo (sub-character) level, which can produce valid but altered syllables or expose raw jamo, thereby disrupting tokenization and bypassing standard error correction. By applying five jamo-level perturbations to the KMMLU benchmark, the authors show that model accuracy degrades steadily with perturbation intensity and that larger models do not gain robustness. They further demonstrate that corrupted inputs shift internal representations in a detectable way, enabling a linear probe to identify unseen typos and motivate a Typo-Aware Chain-of-Thought (TACoT) strategy that selectively triggers chain-of-thought inference only when a typo is detected, recovering much of the accuracy benefit at lower cost.
By Seojin Lee, Hwanhee Lee
arXiv:2610.31723v1 Announce Type: new
Abstract: This article introduces Latin Modern BERT (LAMB), an encoder for the Latin language based on the ModernBERT architecture, pretrained in two phases on m...
By Aidan McCarthy