arXiv Computation and Language October 16, 2026 By Chen Hu, Yintao Tai, Antonio Vergari, Frank Keller, Alessandro Suglia

MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts

Read the original on arXiv Computation and Language →

MIXAR is the first generative multilingual pixel‑based language model, pretrained on eight high‑resource languages across Latin and CJK scripts with up to 0.5 B parameters. It replaces unstable adversarial training with a GRPO‑based reinforcement‑learning stage, improving free‑generation accuracy by up to 2.3× on multilingual benchmarks. MIXAR generalizes better to unseen low‑resource languages than a tokenizer‑based model trained on the same corpus, matches tokenizer models on discriminative tasks, and resists orthographic attacks more gracefully.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Sep 1

Seeing the Unseen: Visual Similarity for Pixel Language Model Adaptation

arXiv:2608.30541v1 Announce Type: new Abstract: Pixel-based language models (LMs) replace traditional tokenizers by processing rendered images of text, making cross-lingual transfer heavily dependent...

By Ran Zhang, Miryam de Lhoneux, Wessel Poelman

llms

arXiv Computer Vision

Sep 10

Let ViT Speak: Generative Language-Image Pre-training

arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...

By Yan Fang, Mengcheng Lan, Zilong Huang, Weixian Lei, Yunqing Zhao, Yujie Zhong, Yingchen Yu, Qi She, Yao Zhao, Yunchao Wei

llms computer-vision multimodal benchmarks

arXiv Computation and Language

Sep 2

On the Design Fundamentals of Pixel Text Representation Learning

arXiv:2609.01147v1 Announce Type: cross Abstract: Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders strug...

By Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang, Yu Rong, Hong Cheng, Hou Pong Chan, Chenghao Xiao

multimodal benchmarks safety

arXiv AI

Oct 7

VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs

VisionWeave introduces elastic visual representation weaving, a native capability for multimodal large language models that learns where and at what granularity to encode visual information. The method combines a gated spatial pooler for coarse representations with a granularity router that allocates content‑adaptive token usage, trained end‑to‑end on large‑scale data. Experiments on Qwen3.5‑4B and Qwen3.8‑27B show that VisionWeave can save 43.0% of tokens while preserving 98.9% of performance across eight benchmarks, and delivers significant throughput gains and latency reductions when deployed on the SGLang serving engine.

By Yuan Feng, Qize Yang, Ruizhe Chen, Sibo Song, Haolin He, Muzhi Zhu, Zihan Liu, Yunfei Chu, Xize Cheng, Yuxuan Wang, Jin Xu, Xike Xie

llms efficiency multimodal benchmarks

arXiv AI

Sep 3

NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference

NeoMME is a family of 260M and 800M‑parameter multimodal‑native multilingual encoders that process text and raw image patches in a single bidirectional Transformer. Trained from scratch with a masked discrete‑diffusion objective conditioned on visible image patches, NeoMME supports a 16,384‑token context, enabling encoding of up to two 4K UHD images. In downstream tests, NeoMME‑Retriever models outperform all sub‑800M‑parameter baselines on the ViDoRe v3 benchmark and achieve twice the throughput of ColModernVBERT on an NVIDIA L40S, while hierarchical token pooling and asymmetric quantization compress embeddings 255× with minimal loss in retrieval performance.

By Aur\'elien Lac, Tony Wu

llms rag diffusion fine-tuning efficiency multimodal benchmarks

arXiv Machine Learning

Aug 11

Embedding Initialization for Unseen Low-resource Languages in Multilingual NMT: A Case Study on Limbum-English Translation

arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.

By Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu, Marvin Ogore, Samuel Oluwajunwonlo Babalola, Oche Ankeli

llms rag nlp fine-tuning