Hugging Face Trending Papers
DPA-MoE: A Runtime Precision Control Framework for MoE Inference
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
Oct 16
DPA-MoE: A Runtime Precision Control Framework for MoE Inference
DPA-MoE is a runtime precision control framework that allows Mixture-of-Experts (MoE) models to use any-precision routed experts. It encodes expert feed‑forward weights as prefix‑decodable chunks, profiles reconstruction residuals offline, and allocates chunks during inference based on live router gate scores. This enables a single encoded MoE checkpoint to operate at multiple effective weight budgets without retraining, improving KL divergence and downstream accuracy on Qwen3‑30B‑A3B and DeepSeekMoE‑16B with modest throughput overhead.
arXiv Machine Learning
Sep 14
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Dynamic Expert Quantization (DynaExq) is a runtime-aware mixed-precision serving system designed for single‑GPU Mixture‑of‑Experts (MoE) inference under a hard high‑bandwidth memory (HBM) envelope. It treats the problem as an online, budget‑constrained precision allocation task, keeping the most frequently used experts at higher precision while relegating the rest to low‑precision fallbacks. By estimating expert hotness from router traces and asynchronously promoting or demoting experts, DynaExq maintains a fully materialized expert set during the forward pass, improving accuracy and throughput compared to static post‑training quantization and offloading/prefetch baselines. whyItMatters":"DynaExq enables efficient deployment of large MoE models on memory‑limited GPUs by dynamically allocating precision based on runtime expert usage, thereby reducing memory footprint and latency while boosting accuracy and throughput."
Hugging Face Trending Papers
Jul 27
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU.
arXiv AI
Jul 28
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
arXiv:2607. 24434v1 Announce Type: cross Abstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory.
arXiv AI
Sep 7
ACE: Adaptive Calibration-Free Expert Skipping for MoE-based LLMs
ACE introduces a training‑free, calibration‑free framework for adaptive expert skipping in Mixture‑of‑Experts LLMs. It combines a Global Spectral Proxy that estimates global transformation capacity with a Router‑Conditioned Refinement that builds expert‑specific direction prototypes, enabling the model to skip low‑contribution experts while always keeping the top‑1 expert. Offline computation of expert statistics leaves only lightweight table lookups during inference, and experiments on three MoE‑based LLMs show ACE outperforms static and dynamic baselines, especially at high skipping ratios.
arXiv Machine Learning
Aug 27
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.