arXiv Machine Learning June 16, 2026 By Zongfang Liu, Jinghui Zhang, Zijian Ma, Guangyi Chen, Xin Yuan

How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle

Read the original on arXiv Machine Learning →

arXiv:2606. 15716v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models reduce per-token computation through sparse expert activation, yet deployment still requires storing the full expert pool, making one-shot expert pruning a practical approach for reducing memory usage.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 17

AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning

arXiv:2603. 18492v3 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making expert pruning important for reducing memory and serving overhead.

By Zongfang Liu, Guangyi Chen, Shengkun Tang, Yifan Shen, Huan Wang, Xin Yuan

llms efficiency benchmarks

Hugging Face Trending Papers

Jul 2

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

Sparsely activated Mixture-of-Experts (MoE) language models contain substantial structured redundancy among routed experts, but pruning them without downstream calibration data remains challenging. Existing expert-pruning methods typically rely on a single aggregated importance score, which can bias the retained set toward experts favored by dominant calibration patterns.

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Less is MoE: Trimming Experts in Domain-Specialist Language Models

arXiv:2606. 05538v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models achieve strong performance through conditional computation, but their large parameter footprint poses deployment challenges.

By Haoze He, Xinkai Zou, Xuan Jiang, Xingyuan Ding, Ao Qu, Juncheng Billy Li, Heather Miller

llms efficiency benchmarks

arXiv AI

Jul 3

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

arXiv:2607. 01710v1 Announce Type: new Abstract: Sparsely activated Mixture-of-Experts (MoE) language models contain substantial structured redundancy among routed experts, but pruning them without downstream calibration data remains challenging.

By Yongqin Zeng, Sicheng Pan, Jiale Wang, Hai-tao Zheng, Hong-Gee Kim, Chunxia Ma, XiuTeng Zhou

llms efficiency benchmarks safety

arXiv AI

Jun 2

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts

arXiv:2606. 01062v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performance remains a challenge.

By Jiarui Feng, Hanqing Zeng, Karish Grover, Ruizhong Qiu, Yinglong Xia, Qiang Zhang, Qifan Wang, Ren Chen, Dongqi Fu, Jiayi Liu, Zhoukai Zhao, Xiangjun Fan, Benyu Zhang, Yixin Chen

llms fine-tuning

arXiv Machine Learning

Jun 29

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.

By Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

llms fine-tuning efficiency

Related stories

AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

Less is MoE: Trimming Experts in Domain-Specialist Language Models

Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models