Mixtral of experts

Mixture of Experts Explained

Machine Learning Experts - Sasha Luccioni

Mixture of Experts (MoEs) in Transformers

Machine Learning Experts - Margaret Mitchell

Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains

PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization

arXiv:2606. 03444v1 Announce Type: cross Abstract: Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation.

Machine Learning Experts - Lewis Tunstall

Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA

arXiv:2607. 26052v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts $k$.

Profy: Interpretable Visualization of Expertise-Dependent Motor Skills Toward Supporting Piano Practice

arXiv:2606. 10627v1 Announce Type: cross Abstract: The quality of piano performance depends on nuanced timing, articulation, and dynamic control, but practice feedback is often summary-based and hard to act on.

Expert Support case study: Bolstering a RAG app with LLM-as-a-Judge

Robustness of Mixtures of Experts to Feature Noise

arXiv:2601. 14792v2 Announce Type: replace Abstract: Despite their practical success, it remains unclear why Mixture of Experts (MoE) models can outperform dense networks beyond sheer parameter scaling.

Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations

arXiv:2607. 20426v1 Announce Type: cross Abstract: Existing LLM hallucination mitigation methods, including prompt engineering and model optimization, either hardly alter models'internal knowledge or have poor cross-domain generalization.