arXiv Machine Learning June 2, 2026 By Adrian Zhao, Zhenkun Cai, Zhenyu Song, Lingfan Yu, Haozheng Fan, Jun Wu, Yida Wang, Nandita Vijaykumar

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving

Read the original on arXiv Machine Learning →

arXiv:2603. 28768v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) has recently emerged as the mainstream architecture for efficiently scaling large language models while maintaining near-constant computational cost.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference

arXiv:2608. 07964v1 Announce Type: cross Abstract: Load Balancing has emerged as a critical problem in expert-parallel distributed inference of Mixture-of-Experts (MoE) models.

By Yize Wu, Ke Gao, Ling Li, Yanjun Wu

arXiv AI

Jun 18

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

arXiv:2606. 19025v1 Announce Type: cross Abstract: Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators.

By Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji, Alex Iacob, Andrej Jovanovic, Yan Gao, Wanru Zhao, Nicholas D. Lane

llms efficiency benchmarks

arXiv AI

3d ago

MoEless: Efficient MoE LLM Serving with Serverless Experts

arXiv:2603.06350v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly adopt Mixture-of-Experts (MoE) architectures to scale efficiently under stringent resource constrai...

By Hanfei Yu, Bei Ouyang, Shwai He, Ang Li, Hao Wang

llms efficiency benchmarks

arXiv AI

Jul 14

HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference

arXiv:2607. 11586v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) large language models (LLM) activate only a small number of experts during inference, but token routing introduces persistent expert hotness skew: a small set of hot experts continuously receives most tokens, while the remaining experts are lightly loaded.

By Yongqin Zhang

llms

arXiv Machine Learning

5d ago

TeDiServe: High SLO Attainment Serving for Diffusion Language Models

TeDiServe is a cluster‑level serving system designed for diffusion language models (DLMs). It addresses DLM‑specific challenges such as the speed‑quality tradeoff from confidence‑based denoising, variable parallelization under fluctuating load, and non‑uniform per‑step costs from approximate KV caching. By employing deadline‑aware scheduling, adaptive load control, and a quality‑aware optimization for cluster reconfiguration, TeDiServe achieves up to 56.6 percentage points higher SLO attainment and reduces end‑to‑end latency by up to 46% with less than 1% accuracy loss.

By Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham, Shivaram Venkataraman

llms diffusion efficiency benchmarks

arXiv AI

3d ago

TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training

arXiv:2604.09107v2 Announce Type: replace-cross Abstract: Modern LLM reinforcement learning (RL) workloads require a high-performance weight transfer system to scale training across heterogeneous com...

By Chenhao Ye, Huaizheng Zhang, Mingcong Han, Baoquan Zhong, Xiang Li, Qixiang Chen, Xinyi Zhang, Weidong Zhang, Kaihua Jiang, Wang Zhang, He Sun, Wencong Xiao, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau

llms reinforcement-learning

One email a morning, machine-written

Related stories

EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

MoEless: Efficient MoE LLM Serving with Serverless Experts

HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference

TeDiServe: High SLO Attainment Serving for Diffusion Language Models

TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training