← Back to all news
arXiv Computer Vision By Pengcheng Zheng, Chaoning Zhang, Jiaxin Yan, Sihan Cao, Jianwei Zhang, Xudong Wang, Jiaquan Zhang, Jewon Lee, Tae-Ho Kim, Yang Yang, Heng Tao Shen

MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models

Read the original on arXiv Computer Vision →

MoR-MLLM introduces a computation-sparse multimodal large language model that uses adaptive per-token recursion to dynamically adjust recursive depth. This approach allocates more computation to visually or linguistically challenging tokens while skipping redundant operations for simpler ones. A three-stage MoR-Tuning strategy and entropy-regularized loss stabilize training, enabling the model to reduce training memory and computation complexity while maintaining high performance on vision-language tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Oct 2

Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation

The paper investigates whether the sparsity of Mixture-of-Experts (MoE) models leads to intrinsic semantic organization across modalities and domains. It shows that experts naturally specialize semantically even without explicit modular training. The authors propose ExpertLens, a data‑free method that decodes router weights to identify domain‑specialized experts, enabling selective fine‑tuning that matches or exceeds full fine‑tuning while updating only 21.7–47.0% of parameters and achieving a 4.0× speedup, outperforming LoRA in both performance and efficiency.

By Damiano Marsili, Raphi Kang, Aditya Mehta, Pietro Perona, Georgia Gkioxari
fine-tuningefficiencymultimodal
More like this →
arXiv Computation and Language
5d ago

Scaling Native Multimodal Pre-Training From Scratch

The paper investigates how to scale a Transformer-based vision‑language model trained from scratch on multimodal inputs under a fixed compute budget. It finds that minimal loss follows a predictable compute law while optimal model size and token count obey power‑law scaling, with language and multimodal objectives showing distinct allocation trends. The study also proposes a budget‑compensation rule that allows additional multimodal tokens to offset text‑efficiency penalties, and demonstrates that native multimodal pre‑training improves spatial reasoning and few‑shot learning.

By Haoyuan Wu, Aoqi Wu, Hai Wang, Jiajia Wu, Jinxiang Ou, Bei Yu
llmsmultimodal
More like this →
arXiv AI
Jun 26

TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference

arXiv:2606. 27161v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of visual tokens, which introduces substantial computational overhead.

By Tinghao Wang, Yichen Guo, Rui Huang, Zheng Lu, Qizhe Zhang, Chenxi Li, Yuan Zhang, Jiajun Cao, Zhirong Shen, Yaosong Du, Guangyan Gan, Wenya Wang, Lin William Cong, Shanghang Zhang
llmsefficiencymultimodalbenchmarkssafety
More like this →