arXiv Machine Learning October 23, 2026 By Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le, Hoang Phan, Thien Huu Nguyen, Linh Ngo Van, Trung Le

Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs

Read the original on arXiv Machine Learning →

Route-and-Generate (Rouge) is a hierarchical training framework that lets large language models decide how much reasoning to apply to each input. It separates reasoning effort from answer generation by using a routing policy that distributes a fixed sampling budget across different reasoning formats, while the base model is trained with GRPO and per-sample rewards. The method employs a rank‑based reward favoring shorter correct solutions and updates the router through a closed‑form KL‑regularized policy improvement, leading to more concise answers on easy tasks and longer reasoning on harder ones, thereby improving accuracy‑efficiency trade‑offs on multiple reasoning benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language

Sep 22

Rubric-Guided Process Reward for Stepwise Model Routing

arXiv:2605.29310v2 Announce Type: replace-cross Abstract: Stepwise model routing improves the efficiency of Large Reasoning Models (LRMs) by assigning each reasoning step to a suitable model. Recent...

By Shenghao Ye, Yu Guo, Zhengheng Li, Shuangwu Chen, Jian Yang

reinforcement-learning benchmarks

arXiv AI

Aug 11

Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning in Agents

arXiv:2604. 05164v3 Announce Type: replace-cross Abstract: As LLM reasoning performance plateaus, improving inference-time compute efficiency is crucial to mitigate overthinking and long thinking traces even for simple queries.

By Neharika Jali, Anupam Nayak, Gauri Joshi

llms agents reinforcement-learning benchmarks

arXiv AI

Sep 18

When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models

When2Think introduces a post‑training framework that dynamically allocates reasoning depth in Large Reasoning Models based on instance difficulty. The method uses Instance‑level Difficulty‑Aware Control (IDAC) to shape rewards with pre‑computed accuracy and token usage statistics, enabling stable, critic‑free optimization without learned reward models. Experiments on mathematical benchmarks show that When2Think improves accuracy‑efficiency trade‑offs, achieving higher Pass@3 scores while reducing token usage compared to baseline models.

By Jaejun Shim, HyunJin Kim, Young Jin Kim, JinYeong Bak

benchmarks

arXiv Machine Learning

Jun 5

IR3DE: A Linear Router for Large Language Models

arXiv:2606. 06098v1 Announce Type: cross Abstract: Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs.

By Eros Fan\`i, O\u{g}uzhan Ersoy

llms

arXiv Machine Learning

Sep 22

Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving

arXiv:2609.23085v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are creating rapidly growing inference energy demands as model sizes grow and reasoning trajector...

By Muhammad Abdur Rab Siddiqui, Daniela Rojas, Chen Yang, Wenqi Cui, Yuanyuan Shi, Yize Chen

llms agents fine-tuning benchmarks

arXiv Computation and Language

Oct 5

Hint-Guided Diversified Policy Optimization for LLM Reasoning

arXiv:2606.03021v4 Announce Type: replace Abstract: Recent developments in Large Language Models (LLMs) have showcased impressive reasoning capabilities, with Reinforcement Learning with Verifiable R...

By Zhiyu Cao, Kaixin Wu, Mingjie Zhong, Peifeng Li, Can Ye, Qiaoming Zhu

llms reinforcement-learning