arXiv AI
MoEless: Efficient MoE LLM Serving with Serverless Experts
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 11
EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
arXiv:2608. 07964v1 Announce Type: cross Abstract: Load Balancing has emerged as a critical problem in expert-parallel distributed inference of Mixture-of-Experts (MoE) models.
arXiv AI
Aug 17
FreeBalance: Pre-Routing Online Moe Load Balancing via Residual Workload Prediction
arXiv:2608. 14205v1 Announce Type: new Abstract: Load imbalance poses a major bottleneck to the efficiency of expert parallelism in distributed inference of Mixture-of-Experts (MoE) models.
arXiv Machine Learning
Jun 16
A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference
arXiv:2606. 15453v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) based large language models (LLMs), such as Qwen and DeepSeek, have recently emerged as an effective approach to improving model capacity without proportionally increasing computational cost.
arXiv Machine Learning
Aug 11
MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
arXiv:2510. 19366v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs).
arXiv AI
Jul 13
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
arXiv:2607. 08782v1 Announce Type: cross Abstract: Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models.
arXiv AI
Jul 29
SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models
arXiv:2607. 24787v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models expand foundation model capacity through conditional expert activation, but their full expert pools remain difficult to deploy under limited accelerator memory.