arXiv Machine Learning October 28, 2026 By Hanming Zhang, Erhu Feng

COPE: Composable Performance Emulation for Mixture-of-Experts Offloading Policies

Read the original on arXiv Machine Learning →

COPE (Composable Offloading Policy Emulator) is a system that predicts the best mixture‑of‑experts (MoE) offloading policy for a given model‑platform configuration before runtime integration. It uses a Policy Matrix to separate expert residency and execution strategy, and a Virtual Heterogeneous Machine to profile combined‑operator schedules and estimate performance from offline routing traces. Compared to baselines, COPE improves fastest‑policy selection accuracy by 24‑100 % and reduces prediction error by 1.7‑6.8×, cutting policy‑specific implementation effort by up to 74 %.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 30

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-...

By Sanjali Yadav, Bahar Asgari

efficiency benchmarks

arXiv AI

Jul 29

SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models

arXiv:2607. 24787v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models expand foundation model capacity through conditional expert activation, but their full expert pools remain difficult to deploy under limited accelerator memory.

By Jinwei Kong, Runqi Meng, Fanyi Wang, Wentao Qiu, Haotian Hu, Yongjian Zhou, Zhenhua Ge

efficiency benchmarks

arXiv Machine Learning

Jun 2

ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving

arXiv:2606. 00735v1 Announce Type: cross Abstract: In distributed Mixture-of-Experts (MoE) inference, input-dependent token routing interacts with GPU performance variability to create persistent stragglers under synchronized execution, where the slowest GPU determines layer latency.

By Seokjin Go, Marko Scrbak, Ephrem Wu, Srilatha Manne, Divya Mahajan

llms efficiency

arXiv Machine Learning

Aug 11

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs

arXiv:2510. 19366v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs).

By Xinfeng Xia, Xiaofeng Hou, Jiacheng Liu, Wenfeng Wang, Mingxuan Zhang, Peng Tang, Chao Li, Minyi Guo

llms efficiency

arXiv Machine Learning

Aug 5

AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding

arXiv:2608. 02989v1 Announce Type: new Abstract: Speculative decoding verifies a tree of draft tokens in one target-model forward pass.

By Shuang Liang (Mark), Hao (Mark), Chen, Zhiwen Mo, Qianzhou Wang, Guoyu Li, Lingxiao Ma, Wayne Luk

efficiency benchmarks

arXiv AI

Oct 15

ConMoE: MoE Compression as Expert Selection and Functional Assignment

arXiv:2605.29350v2 Announce Type: replace Abstract: Sparse Mixture-of-Experts (MoE) models scale model capacity through sparse activation, but storing their full parameters remains costly in GPU memo...

By Yilun Yao, Jiaming Pan, Elsie Dai, Peizhuang Cong, Chao-Hsuan Liu, Yaoming Li, Tong Yang

efficiency benchmarks

One email a morning, machine-written

Related stories

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models

ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs

AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding

ConMoE: MoE Compression as Expert Selection and Functional Assignment