← Back to all news
arXiv Computation and Language By Guanzhi Deng, Zengyan Liu, Haibo Wang, Kuan Wu, Shing Yin Wong, Sichun Luo, Zhuoran Wang, Linqi Song

A Strong MoE Is All You Need for Process Supervision

Read the original on arXiv Computation and Language →

The paper demonstrates that expert routing statistics in Mixture-of-Experts (MoE) models can serve as an intrinsic process-confidence signal, rivaling conventional output-logit confidence. Leveraging this insight, the authors develop RoCo-Rethink, which selectively regenerates uncertain reasoning steps, and RoCo-PRM, which automatically annotates reasoning trajectories using routing-derived step scores and distills them into a standalone process reward model. Across various MoE architectures and reasoning tasks, both methods outperform strong baselines and transfer effectively between models and tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
2d ago

Routes of Reasoning: Reading and Reshaping Test-Time Compute in MoE Models

The paper introduces a method to control reasoning behaviors in Mixture-of-Experts (MoE) language models by labeling sentence-level segments with six distinct reasoning behaviors (planning, execution, restatement, fact, reflection, and conclusion). Linear probes on MoE router logits can predict these behaviors nearly as accurately as probes on the full hidden state, enabling the authors to edit router scores during inference to reduce overthinking. This intervention cuts generated tokens by a median of 30.4% (up to 63.3%) across multiple models and benchmarks without significant loss in accuracy, and often improves it.

By Shengjie Wang, Mohsen Fayyaz, Nanyun Peng
benchmarks
More like this →
arXiv Computation and Language
Sep 30

RAWR: Reward Assignment Without Rollouts in Verifiable Domains

arXiv:2603.17815v2 Announce Type: replace Abstract: Understanding and evaluating multi-step reasoning in LLMs at the level of individual steps remains a key challenge. Process reward models (PRMs) pr...

By Corentin Royer (International Business Machines), Anna Hedstr\"om (ETH AI Center), Debarun Bhattacharjya (Lirio), Gaetano Rossiello (International Business Machines), Andrea Giovannini (International Business Machines), Mennatallah El-Assady (Department of Computer Science, ETH Zurich)
llmsbenchmarksdata-engineering
More like this →
arXiv Machine Learning
Oct 23

Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs

Route-and-Generate (Rouge) is a hierarchical training framework that lets large language models decide how much reasoning to apply to each input. It separates reasoning effort from answer generation by using a routing policy that distributes a fixed sampling budget across different reasoning formats, while the base model is trained with GRPO and per-sample rewards. The method employs a rank‑based reward favoring shorter correct solutions and updates the router through a closed‑form KL‑regularized policy improvement, leading to more concise answers on easy tasks and longer reasoning on harder ones, thereby improving accuracy‑efficiency trade‑offs on multiple reasoning benchmarks.

By Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le, Hoang Phan, Thien Huu Nguyen, Linh Ngo Van, Trung Le
llmsbenchmarks
More like this →