arXiv Machine Learning August 27, 2026 By Juntong Wu, Yifei Liu, Junyi Chen, Siqi Fan, Chaoran Feng, Minghao Li, Liujie Zhang, Weihang Chen, Li Yuan

ExFold: Unified Expert Folding for Training-Free MoE Prefill-Decode Acceleration

Read the original on arXiv Machine Learning →

ExFold is a training‑free expert‑folding framework that jointly accelerates the prefill and decode phases of Mixture‑of‑Experts (MoE) models by projecting the contributions of excluded experts onto a retained expert set using calibrated scalar projectors. It treats both phases as a budgeted output‑approximation problem, achieving token‑level Top‑K folding for prefill and batch‑level expert‑pool folding for decode. Implemented as a plug‑and‑play plugin in vLLM with a lightweight CUDA kernel, ExFold delivers up to 1.41× TTFT and 2.45× TPOT speedups while preserving about 99% of the original model quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference

arXiv:2511. 04805v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models have shown strong potential in scaling language models efficiently by activating only a small subset of experts per input.

By Yushu Zhao, Zheng Wang, Minjia Zhang

llms benchmarks

arXiv Machine Learning

Jun 29

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.

By Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

llms fine-tuning efficiency

arXiv AI

Aug 17

DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding

arXiv:2608. 14385v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models have been widely adopted in real-time interactive applications such as coding assistants, real-time audio-video interaction systems.

By Zewen Jin, Shen Fu, Zeping Duan, Shannon Wang, Weihao Wu, Chengjie Tang, Congkun Ai, Ping Gong, Zijian Dai, Youhui Bai, Cheng Li

llms

arXiv Machine Learning

Jun 17

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models

arXiv:2602. 06154v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) models scale large language models efficiently by sparsely activating experts, but once an expert is selected, it is executed fully.

By Nurbek Tastan, Stefanos Laskaridis, Karthik Nandakumar, Samuel Horvath

llms benchmarks

arXiv Computation and Language

2d ago

PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Composition

arXiv:2609.01024v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. Howeve...

By Ziyan Gan, Fangxin Liu, Chenyang Guan, Junjie Wang, Ning Yang, Haomin Li, Xiang Li, Siran Yang, Jiamang Wang, Lin Qu, Zongwu Wang, Li Jiang, Haibing Guan

llms efficiency

arXiv AI

Jul 9

TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation

arXiv:2607. 06601v1 Announce Type: cross Abstract: Conditional computation can decouple language model quality from per-token inference cost, yet leading techniques act on a single axis in isolation: Mixture-of-Experts (MoE) sparsifies the FFN, Mixture-of-Depths (MoD) skips whole transformer blocks, and KV-cache quantization compresses attention memory.

By Andrii Balashov, Olena Ponomarova

llms efficiency

One email a morning, machine-written

Related stories

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models

PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Composition

TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation