The paper examines whether expert‑routing statistics from a Mixture‑of‑Experts language model can inform layer‑wise mixed‑precision quantization of routed‑expert feed‑forward networks. By building a routing‑capture pipeline for Qwen‑3.6‑35B‑A3B, the authors derive a composite sensitivity score and use coordinate descent to allocate bits per layer under a fixed budget. Experiments across flat, steep, and random allocation regimes show that routing‑guided reallocation does not outperform a uniform baseline, suggesting that routing behavior is an unreliable proxy for true quantization cost.
By Yifan Liu
arXiv:2608. 08627v1 Announce Type: new Abstract: Sparse mixture-of-experts (MoE) layers expand recommendation capacity through conditional computation, yet a trained checkpoint still stores and routes over its full expert bank.
By Lei Xin, Bin Gu, Peize Li, Zitong Wang, Jianbo Zhao, Changjiang Jiang, Yanyue Xie, Chao Huang, Xuyang Zhao, Zunhai Su, Fanhu Zeng, Zhenglun Kong
arXiv:2510. 19366v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs).
By Xinfeng Xia, Xiaofeng Hou, Jiacheng Liu, Wenfeng Wang, Mingxuan Zhang, Peng Tang, Chao Li, Minyi Guo
Dynamic Expert Quantization (DynaExq) is a runtime-aware mixed-precision serving system designed for single‑GPU Mixture‑of‑Experts (MoE) inference under a hard high‑bandwidth memory (HBM) envelope. It treats the problem as an online, budget‑constrained precision allocation task, keeping the most frequently used experts at higher precision while relegating the rest to low‑precision fallbacks. By estimating expert hotness from router traces and asynchronously promoting or demoting experts, DynaExq maintains a fully materialized expert set during the forward pass, improving accuracy and throughput compared to static post‑training quantization and offloading/prefetch baselines.
whyItMatters":"DynaExq enables efficient deployment of large MoE models on memory‑limited GPUs by dynamically allocating precision based on runtime expert usage, thereby reducing memory footprint and latency while boosting accuracy and throughput."
By Kexin Chu, Dawei Xiang, Zixu Shen, Yiwei Yang, Zecheng Liu, Wei Zhang
The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.
By Parivesh Priye, Yufeng Wang, Haibin Ling, Michael Chaykowsky
arXiv:2609.36222v1 Announce Type: new
Abstract: Large language models are increasingly expensive to serve. In large-scale serving systems, autoregressive decoding is often bottlenecked by transferrin...
By Ali Abbasi, Justin Shi, Soheil Kolouri
arXiv:2607. 08782v1 Announce Type: cross Abstract: Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models.
By Qianli Liu, Kaibin Guo, Zicong Hong, Peng Li, Fahao Chen, Haodong Wang, Jian Lin, Song Guo
arXiv:2608. 12146v1 Announce Type: cross Abstract: Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks.
By Yibo Shen, Xudong Han, Xiaowei Zhu, Gen Li, Zhenxuan Pan
LRMoE is a memory-aware training system for Sparse Mixture-of-Experts models that recalibrates backward demand and balances workload to avoid GPU memory bottlenecks. It plans retention quotas and expert placement during forward passes, then uses a load recalibration metric in backpropagation to select token‑expert pairs within these quotas. The system compacts communication payloads and employs variable‑size grouped GEMMs, cutting backward traffic, computation, and buffer usage, achieving 21.2‑44.8% faster training on large models compared to SYMI.
By Jiaming Hu, Zhida Jiang, Zhaolong Xing, Yu Li, Zhen Chen, Ke Zhang
arXiv:2608. 09250v1 Announce Type: new Abstract: Federated learning (FL) must serve devices with varying computational capabilities.
By Bostan Khan, Masoud Daneshtalab
ProQuant is a weight‑only post‑training quantization framework designed for Mixture‑of‑Experts large language models. It introduces PoP, which jointly prunes experts and assigns mixed‑precision bit‑widths to better use a tight bit budget, and RoCo, a routing‑consistent RMSNorm fine‑tune that mitigates routing drift caused by quantization errors. Experiments show ProQuant outperforms existing methods on a range of generative tasks.
arXiv:2606. 09886v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing.
By Yuhao Zhang