arXiv AI
Jun 15
Token-Level LLM Collaboration via FusionRoute
arXiv:2601. 05106v5 Announce Type: replace Abstract: Large language models (LLMs) exhibit strengths across diverse domains.
DLLG: Dynamic Logit-Level Gating of LLM Experts
Leveraging multiple specialized LLMs can combine complementary strengths, but existing approaches trade adaptability for stability: routing commits prematurely, heuristic ensembling depends on fragile proxies, and parameter merging introduces interference. We propose DLLG (Dynamic Logit-Level Gating), a dynamic logit-level ensembling framework that learns token-level expert fusion from sparse response-level supervision.
Related stories
arXiv AI
Jun 4
Adaptive Minds: Empowering Agents with LoRA-as-Tools
arXiv:2510. 15416v2 Announce Type: replace Abstract: We investigate a framework in which LoRA adapters are treated as callable tools that a base language model can dynamically select and invoke.
arXiv AI
Jun 26
Tuning Language Models by Mixture-of-Depths Ensemble
arXiv:2410. 13077v2 Announce Type: replace-cross Abstract: Transformer-based Large Language Models (LLMs) traditionally rely on final-layer loss for finetuning and final-layer representations for predictions, potentially overlooking the predictive power embedded in late layers.
arXiv AI
Aug 10
TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation
arXiv:2608. 06396v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream adaptation.
arXiv AI
Jun 3
EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
arXiv:2606. 03108v1 Announce Type: new Abstract: Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static.
arXiv Machine Learning
Jun 5
IR3DE: A Linear Router for Large Language Models
arXiv:2606. 06098v1 Announce Type: cross Abstract: Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs.
arXiv AI
Jul 14
Route, Communicate, and Reason: Gated Routing and Adaptive Depth for Efficient Multi-Agent Reasoning
arXiv:2607. 10836v1 Announce Type: new Abstract: Multi-agent ensembling multiplies active parameters and inference cost without answering three basic questions: which agents to consult, how deeply a query should traverse a hierarchy of agents, and when inter-agent communication is worth its cost.
arXiv AI
Jun 30
Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation
arXiv:2606. 30266v1 Announce Type: cross Abstract: Motion-language agents must possess the bidirectional capability to both understand human movement (motion-to-text, M2T) and generate it from natural language (text-to-motion, T2M).
arXiv Machine Learning
Jul 21
SkillRouter: Skill Routing for LLM Agents at Scale
arXiv:2603. 22455v5 Announce Type: replace Abstract: Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks.
arXiv AI
Jun 2
Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
arXiv:2503. 05641v4 Announce Type: replace-cross Abstract: Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.
arXiv AI
Jul 21
LaT: LLM-as-Trainer for Multi-Task Vehicle Routing Solvers
arXiv:2607. 17708v1 Announce Type: new Abstract: Multi-task neural solvers aim to handle multiple Vehicle Routing Problem (VRP) variants within a unified model, avoiding separate training for each constraint combination.
arXiv AI
Jul 28
LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models
arXiv:2411. 00918v5 Announce Type: replace-cross Abstract: Mixture of experts (MoE) architectures have become a cornerstone for scaling up and are a key component in most large language models such as GPT-OSS, DeepSeek-V3, Llama-4, and Gemini-2.