arXiv:2608.30541v1 Announce Type: new
Abstract: Pixel-based language models (LMs) replace traditional tokenizers by processing rendered images of text, making cross-lingual transfer heavily dependent...
By Ran Zhang, Miryam de Lhoneux, Wessel Poelman
MIXAR is the first generative multilingual pixel‑based language model, pretrained on eight high‑resource languages across Latin and CJK scripts with up to 0.5 B parameters. It replaces unstable adversarial training with a GRPO‑based reinforcement‑learning stage, improving free‑generation accuracy by up to 2.3× on multilingual benchmarks. MIXAR generalizes better to unseen low‑resource languages than a tokenizer‑based model trained on the same corpus, matches tokenizer models on discriminative tasks, and resists orthographic attacks more gracefully.
By Chen Hu, Yintao Tai, Antonio Vergari, Frank Keller, Alessandro Suglia
arXiv:2609.01147v1 Announce Type: cross
Abstract: Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders strug...
By Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang, Yu Rong, Hong Cheng, Hou Pong Chan, Chenghao Xiao
The paper examines OCR adaptation for low‑resource languages, noting that fine‑tuning often hits a performance ceiling in data‑scarce settings. It identifies that lower layers of language‑specific models learn redundant features while higher layers capture script nuances, leading to a structural inefficiency. To address this, the authors propose PSMC, a framework that pre‑trains a base model, specializes it per language, merges the experts via task arithmetic, and co‑trains a unified multilingual backbone, achieving about a 2% improvement in Word Recognition Rate across 10 Indian scripts without adding parameters.
By Achyuth P, Kahaan Shah, Chetan Arora
arXiv:2608. 11002v1 Announce Type: cross Abstract: Text-to-image (T2I) generation has achieved remarkable progress in recent years.
By Sicheng Zhang, Zhonghao Yan, Binzhu Xie, Shi Qiu, Muzammal Naseer, Naveed Akhtar, Mubarak Shah
The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.
By Muge Zhang, Aaron Jencks, Krishna Badikela, Yulia Tsvetkov, Sachin Kumar