arXiv:2510. 15416v2 Announce Type: replace Abstract: We investigate a framework in which LoRA adapters are treated as callable tools that a base language model can dynamically select and invoke.
By Pavan C Shekar, Aswanth Krishnan
arXiv:2503. 05641v4 Announce Type: replace-cross Abstract: Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.
By Justin Chih-Yao Chen, Sukwon Yun, Elias Stengel-Eskin, Tianlong Chen, Mohit Bansal
arXiv:2606. 06098v1 Announce Type: cross Abstract: Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs.
By Eros Fan\`i, O\u{g}uzhan Ersoy
arXiv:2606. 17952v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) architectures enable scaling LLM parameters under a fixed inference budget by activating only a small subset of experts via top-$k$ routing.
By Miko{\l}aj Zasada, {\L}ukasz Struski, Jacek Tabor, Marcin Kurdziel
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.
By Yuanteng Chen, Peisong Wang, Yuantian Shao, Nanxin Zeng, Chang Xu, Jian Cheng
JevSoup is a training‑free framework that separates System One expert routing from System Two execution for low‑rank adaptation (LoRA) models. It selects two experts based solely on input and expert descriptions, retains the leading expert’s update, projects the second onto the orthogonal complement of the first update’s row space, and combines them with equal weights. On 14 PorTAL tasks and three Qwen3 scales, JevSoup improves task‑macro accuracy by up to 1.19 % and sample‑micro accuracy by up to 1.21 % over the strongest evaluated baselines.
By Xiuying Wang, Jiahua Cheng, Shuotian Li, Yufan Cheng, Bowen Deng, Zhexuan Bai, Yichen Li