Hugging Face Trending Papers August 4, 2026

MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation

Read the original on Hugging Face Trending Papers →

Parameter-efficient fine-tuning (PEFT) enables efficient adaptation of large language models, but existing MoE-based PEFT methods typically improve capacity by storing multiple full LoRA experts, causing adapter storage to grow linearly with the number of experts and restricting adaptation to a fixed expert pool. We ask whether MoE-based PEFT can produce instance-specific adaptations without explicitly storing a separate LoRA module for each expert.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jun 2

Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

arXiv:2503. 05641v4 Announce Type: replace-cross Abstract: Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.

By Justin Chih-Yao Chen, Sukwon Yun, Elias Stengel-Eskin, Tianlong Chen, Mohit Bansal

llms agents benchmarks

arXiv AI

Jul 1

Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

arXiv:2606. 31413v1 Announce Type: new Abstract: Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared.

By Seyed Alireza Molavi, Zhan Su, Yan Hu, Peyman Sheikholharam Mashhadi, Stefan Byttner, Prayag Tiwari

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Aug 4

Compliance2LoRA: Personalizable On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters

arXiv:2607. 27594v2 Announce Type: replace Abstract: Post-training alignment in large reasoning models (LRMs) has significantly improved their adaptability to diverse safety compliance settings.

By Pankayaraj Pathmanathan, Furong Huang

fine-tuning safety

arXiv Machine Learning

Jul 31

Compliance2LoRA: On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters

arXiv:2607. 27594v1 Announce Type: new Abstract: Post-training alignment in large reasoning models (LRMs) has significantly improved their adaptability to diverse safety compliance settings.

By Pankayaraj Pathmanathan, Furong Huang

fine-tuning safety

arXiv AI

Jun 2

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts

arXiv:2606. 01062v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have become a leading approach for decoupling parameter count from computational cost in large language models, yet effectively scaling MoE performance remains a challenge.

By Jiarui Feng, Hanqing Zeng, Karish Grover, Ruizhong Qiu, Yinglong Xia, Qiang Zhang, Qifan Wang, Ren Chen, Dongqi Fu, Jiayi Liu, Zhoukai Zhao, Xiangjun Fan, Benyu Zhang, Yixin Chen

llms fine-tuning

arXiv Machine Learning

Jun 2

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models

arXiv:2505. 18877v4 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) lowers the computational and memory overhead of fine-tuning large models by updating a low-dimensional subspace of the pre-trained weight matrix.

By Yilang Zhang, Bingcong Li, Georgios B. Giannakis

llms fine-tuning benchmarks

Related stories

Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

Compliance2LoRA: Personalizable On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters

Compliance2LoRA: On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models