arXiv Machine Learning July 21, 2026 By Anik Jha

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

Read the original on arXiv Machine Learning →

arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 29

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.

By Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

llms fine-tuning efficiency

arXiv Machine Learning

Jul 31

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.

By Jiwon Jang, Kisu Yang, Heuiseok Lim, Hyunwoo Park

llms agents efficiency benchmarks safety

arXiv Machine Learning

Aug 11

Which Decisions Low-Bit Quantization Breaks, and How to Predict Them

arXiv:2608. 06564v2 Announce Type: replace Abstract: Quantization is how large language models are actually deployed, and below four bits it hurts.

By Zekun Wu, Swati Dhiman, Adriano Koshiyama

llms agents efficiency benchmarks safety

arXiv Machine Learning

Aug 10

Quantization Damage Is Multiplicative, Not Additive

arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.

By Zekun Wu, Swati Dhiman, Adriano Koshiyama

llms agents efficiency benchmarks safety

arXiv AI

Jun 10

SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs

arXiv:2606. 09886v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing.

By Yuhao Zhang

llms efficiency benchmarks

arXiv AI

Jun 18

Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression

arXiv:2606. 18304v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale compute efficiently, yet remain expensive to deploy due to their substantial memory footprint and inference overhead.

By Yifu Ding, Jiacheng Wang, Ge Yang, Yongcheng Jing, Jinyang Guo, Xianglong Liu, Dacheng Tao

efficiency benchmarks

Related stories

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

Which Decisions Low-Bit Quantization Breaks, and How to Predict Them

Quantization Damage Is Multiplicative, Not Additive

SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs

Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression