arXiv AI
SLEM: Spectrum-Aware Latent Expert Merging
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 21
Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models
arXiv:2610.27123v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models reduce computation through sparse expert activation, but deployment still requires storing a large number of expert p...
arXiv AI
Aug 11
Shape Mutating Expert Compression:LorExperts and BTExperts
arXiv:2608. 07814v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) language models deliver high capacity at low per-token compute, but deploying them cheaply requires compressing their many expert weight matrices.
arXiv AI
Jun 2
DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
arXiv:2606. 01062v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performance remains a challenge.
arXiv AI
Jun 2
Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
arXiv:2503. 05641v4 Announce Type: replace-cross Abstract: Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.
arXiv AI
Oct 21
Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE
arXiv:2610.27598v1 Announce Type: new Abstract: As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling. However, mainstream model...
arXiv AI
Jul 21
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing
arXiv:2502. 04411v3 Announce Type: replace-cross Abstract: Model merging aggregates Large Language Models (LLMs) finetuned on different tasks into a stronger one.