arXiv AI
DIET: Deletion-response Expert Trimming for Video Diffusion Transformers
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
4d ago
BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding
arXiv:2609.36222v1 Announce Type: new Abstract: Large language models are increasingly expensive to serve. In large-scale serving systems, autoregressive decoding is often bottlenecked by transferrin...
arXiv Machine Learning
1d ago
MoRA: MoE Pruning via Router Bias Learning and Expert Approximation
MoRA is a framework for pruning Mixture-of-Experts (MoE) models by learning a router bias for each expert and optimizing it with a language‑modeling loss and a routing‑diversity regularizer. The learned biases sharpen routing distributions to identify critical experts and encourage diverse routing preferences. After pruning, MoRA uses an expert approximation mechanism that approximates the outputs of pruned experts with affine transformations of remaining experts, further improving performance.
arXiv Computation and Language
6d ago
RAZOR: Pruning Replaceable Experts in LLMs
arXiv:2609.30465v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) models activate few experts per token but store the full expert pool. Expert pruning reduces this storage burden; at a fixed...
arXiv AI
Jun 10
SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
arXiv:2606. 09886v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing.
arXiv Machine Learning
Jul 28
MMOE: Modernizing Diffusion Transformers with Efficient Expert Design
arXiv:2607. 24665v1 Announce Type: cross Abstract: Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.
arXiv Machine Learning
Jun 29
FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models
arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.