The paper demonstrates that expert routing statistics in Mixture-of-Experts (MoE) models can serve as an intrinsic process-confidence signal, rivaling conventional output-logit confidence. Leveraging this insight, the authors develop RoCo-Rethink, which selectively regenerates uncertain reasoning steps, and RoCo-PRM, which automatically annotates reasoning trajectories using routing-derived step scores and distills them into a standalone process reward model. Across various MoE architectures and reasoning tasks, both methods outperform strong baselines and transfer effectively between models and tasks.
arXiv:2605.29310v2 Announce Type: replace-cross Abstract: Stepwise model routing improves the efficiency of Large Reasoning Models (LRMs) by assigning each reasoning step to a suitable model. Recent...
The paper introduces a method to control reasoning behaviors in Mixture-of-Experts (MoE) language models by labeling sentence-level segments with six distinct reasoning behaviors (planning, execution, restatement, fact, reflection, and conclusion). Linear probes on MoE router logits can predict these behaviors nearly as accurately as probes on the full hidden state, enabling the authors to edit router scores during inference to reduce overthinking. This intervention cuts generated tokens by a median of 30.4% (up to 63.3%) across multiple models and benchmarks without significant loss in accuracy, and often improves it.
arXiv:2603.17815v2 Announce Type: replace Abstract: Understanding and evaluating multi-step reasoning in LLMs at the level of individual steps remains a key challenge. Process reward models (PRMs) pr...
Route-and-Generate (Rouge) is a hierarchical training framework that lets large language models decide how much reasoning to apply to each input. It separates reasoning effort from answer generation by using a routing policy that distributes a fixed sampling budget across different reasoning formats, while the base model is trained with GRPO and per-sample rewards. The method employs a rank‑based reward favoring shorter correct solutions and updates the router through a closed‑form KL‑regularized policy improvement, leading to more concise answers on easy tasks and longer reasoning on harder ones, thereby improving accuracy‑efficiency trade‑offs on multiple reasoning benchmarks.
arXiv:2609.22746v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently been introduced into traffic signal control (TSC) as decision agents due to their strengths in human-readabl...
arXiv:2610.35690v1 Announce Type: new Abstract: Serving web queries at scale with multi-agent systems requires jointly configuring large language models, collaboration modes, agent counts, interactio...
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.
arXiv:2605. 02395v2 Announce Type: replace Abstract: Process reward models (PRMs) rely on high-quality process supervision data, yet existing construction methods often provide limited control over error location, error type, and trajectory consistency.
arXiv:2608. 06396v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream adaptation.
The paper introduces Reasoning State Propagation (RSP), a method that models each reasoning prefix with a binary validity state and learns transitions between successive states. RSP predicts break and repair probabilities to connect intermediate reasoning states to the final outcome, enabling outcome supervision to guide learning of earlier steps. Experiments on reasoning search, response selection, and reinforcement learning show RSP consistently outperforms existing Process Reward Models, achieving notable gains over Qwen2.5-Math-PRM.
arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.
The paper introduces Multi-Level Reasoning (MLR), a two-level stochastic framework that separates long chain-of-thought generation into a high-level planner producing structured step descriptors and a low-level executor generating detailed reasoning. MLR replaces single-policy reinforcement learning with an iterative Step‑DPO preference optimization that uses Twisted Sequential Monte Carlo for scalable, stepwise supervision, improving training stability and accuracy. Experiments on math, science, and logic benchmarks show that MLR outperforms both supervised fine‑tuning and strong RL baselines while maintaining robustness over extended reasoning horizons.