arXiv Machine Learning June 17, 2026 By Zongfang Liu, Guangyi Chen, Shengkun Tang, Yifan Shen, Huan Wang, Xin Yuan

AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning

Read the original on arXiv Machine Learning →

arXiv:2603. 18492v3 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making expert pruning important for reducing memory and serving overhead.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 3

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

arXiv:2607. 01710v1 Announce Type: new Abstract: Sparsely activated Mixture-of-Experts (MoE) language models contain substantial structured redundancy among routed experts, but pruning them without downstream calibration data remains challenging.

By Yongqin Zeng, Sicheng Pan, Jiale Wang, Hai-tao Zheng, Hong-Gee Kim, Chunxia Ma, XiuTeng Zhou

llms efficiency benchmarks safety

Hugging Face Trending Papers

Jul 2

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

Sparsely activated Mixture-of-Experts (MoE) language models contain substantial structured redundancy among routed experts, but pruning them without downstream calibration data remains challenging. Existing expert-pruning methods typically rely on a single aggregated importance score, which can bias the retained set toward experts favored by dominant calibration patterns.

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 16

How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle

arXiv:2606. 15716v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models reduce per-token computation through sparse expert activation, yet deployment still requires storing the full expert pool, making one-shot expert pruning a practical approach for reducing memory usage.

By Zongfang Liu, Jinghui Zhang, Zijian Ma, Guangyi Chen, Xin Yuan

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Less is MoE: Trimming Experts in Domain-Specialist Language Models

arXiv:2606. 05538v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models achieve strong performance through conditional computation, but their large parameter footprint poses deployment challenges.

By Haoze He, Xinkai Zou, Xuan Jiang, Xingyuan Ding, Ao Qu, Juncheng Billy Li, Heather Miller

llms efficiency benchmarks

arXiv Machine Learning

Jun 29

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.

By Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

llms fine-tuning efficiency

arXiv Machine Learning

Jul 24

PreMoE: Proactive Inference for Efficient Mixture-of-Experts

arXiv:2505. 17639v4 Announce Type: replace Abstract: Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization.

By Zehua Pei, Ying Zhang, Hui-Ling Zhen, Tao Yuan, Xianzhi Yu, Zhenhua Dong, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu

efficiency

Related stories

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle

Less is MoE: Trimming Experts in Domain-Specialist Language Models

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

PreMoE: Proactive Inference for Efficient Mixture-of-Experts