Hugging Face Trending Papers August 5, 2026

Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models

Read the original on Hugging Face Trending Papers →

Mixture-of-experts vision-language models (MoE-VLMs) increase model capacity with sparse expert activation, yet deployment requires storing the full expert pool. Training-free expert merging reduces this burden, and many routing-based methods aggregate routing statistics across all tokens to determine merge compatibility.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Aug 6

Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models

arXiv:2608. 04454v1 Announce Type: cross Abstract: Mixture-of-experts vision-language models (MoE-VLMs) increase model capacity with sparse expert activation, yet deployment requires storing the full expert pool.

By Hongyu Zhang, Cheng Yan, Xiang Xia, Wuyang Zhang

llms multimodal benchmarks

arXiv Machine Learning

Jun 5

Selective Sinkhorn Routing for Improved Sparse Mixture of Experts

arXiv:2511. 08972v2 Announce Type: replace Abstract: Sparse Mixture-of-Experts (SMoE) models are scalable and computationally efficient, enabling large increases in model capacity with limited inference overhead.

By Duc Anh Nguyen, Huu Binh Ta, Nhuan Le Duc, Tan Minh Nguyen, Toan Tran

computer-vision

arXiv Machine Learning

Aug 26

MoTE: Mixture of Task Experts for Multi-Task Video Understanding

arXiv:2608.24763v1 Announce Type: cross Abstract: Procedural video-language models must solve heterogeneous tasks from the same visual evidence, including action recognition, forecasting, and procedu...

By Muhammad Asad Ali, Umar Khan, Nadia Robertini, Didier Stricker

llms multimodal benchmarks

arXiv AI

Sep 18

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts

The paper introduces L2R, a routing framework for Mixture-of-Experts models that reshapes the routing space into a shared low‑rank latent space and employs Saturated Inner‑Product Scoring to control Lipschitz behavior, resulting in smoother and more stable routing geometry. It also adds a parameter‑efficient multi‑anchor routing mechanism to increase expert expressiveness. Experiments on an OLMoE‑based language model and a ViT‑based ImageNet setting demonstrate improved overall performance and better routing geometry and expert discrimination.

By Minghao Yang, Ren Togo, Guang Li, Takahiro Ogawa, Miki Haseyama

arXiv AI

Aug 10

TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

arXiv:2608. 06396v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream adaptation.

By Guanzhi Deng, Haibo Wang, Kuan Wu, Xiangru Jian, Shing Yin Wong, Sichun Luo, Zhuoran Wang, Linqi Song

llms fine-tuning benchmarks

arXiv Computer Vision

3d ago

DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference

arXiv:2609.38823v1 Announce Type: new Abstract: Multimodal mixture-of-experts (MoE) models combine sparse expert activation with visual-language capabilities, yet their inference remains costly becau...

By Xudong Tan, Peng Ye, Ming Xie, Chenyu Huang, Yaoxin Yang, Jiayuan Fan, Tao Chen

multimodal benchmarks

One email a morning, machine-written

Related stories

Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models

Selective Sinkhorn Routing for Improved Sparse Mixture of Experts

MoTE: Mixture of Task Experts for Multi-Task Video Understanding

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts

TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference