arXiv Machine Learning
DynaNDE: Dynamic Near-Data Expert Scheduling for Batched MoE Inference
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 13
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
arXiv:2607. 08782v1 Announce Type: cross Abstract: Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models.
arXiv Machine Learning
Sep 14
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Dynamic Expert Quantization (DynaExq) is a runtime-aware mixed-precision serving system designed for single‑GPU Mixture‑of‑Experts (MoE) inference under a hard high‑bandwidth memory (HBM) envelope. It treats the problem as an online, budget‑constrained precision allocation task, keeping the most frequently used experts at higher precision while relegating the rest to low‑precision fallbacks. By estimating expert hotness from router traces and asynchronously promoting or demoting experts, DynaExq maintains a fully materialized expert set during the forward pass, improving accuracy and throughput compared to static post‑training quantization and offloading/prefetch baselines. whyItMatters":"DynaExq enables efficient deployment of large MoE models on memory‑limited GPUs by dynamically allocating precision based on runtime expert usage, thereby reducing memory footprint and latency while boosting accuracy and throughput."
Hugging Face Trending Papers
Sep 1
PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Composition
Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. However, modern MoE inference remains heavily constrain...
arXiv Machine Learning
2d ago
Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging
arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-...
arXiv Machine Learning
Jun 16
A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference
arXiv:2606. 15453v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) based large language models (LLMs), such as Qwen and DeepSeek, have recently emerged as an effective approach to improving model capacity without proportionally increasing computational cost.
arXiv Computation and Language
Sep 2
PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Composition
PCoMoE introduces a path‑compositional execution framework that moves Mixture‑of‑Experts inference from coarse‑grained expert selection to fine‑grained path composition. It uses a path‑level formulation of expert computation, a compatibility‑aware layer‑wise pruning strategy to eliminate low‑value path combinations, and a hardware‑friendly execution engine that reuses sub‑expert structures with bounded overhead. Experiments show up to a 1.31× speedup and a 10% accuracy improvement over existing MoE inference methods.