AdaThinking-E introduces a reinforcement learning framework that uses one-token entropy regulation to enable large language models to decide adaptively whether to engage in deep reasoning. By measuring entropy in the predicted probability distribution at key decision tokens, the model learns to explore different thinking strategies during training and converge to confident, efficient decision policies. Experiments show the method improves accuracy on complex tasks while reducing computational overhead on simpler ones across various document reasoning benchmarks.
By Zining Wang, Tongkun Guan, Boming Chen, Zhentao Guo, Jianqiang Liu, Chao Jin, Chen Duan, Kai Zhou, Pengfei Yan, Wei Shen, Xiaokang Yang
PACIFIER is a graph‑learning framework that generates intervention sequences to reduce opinion polarization in online social networks modeled by the Friedkin‑Johnsen framework. It uses history‑aware node representations and both greedy and reinforcement‑learning strategies to score actions, supporting multiple moderation types and continuous opinions. Trained on small synthetic graphs, PACIFIER transfers effectively to large real‑world Twitter networks, outperforming baselines by up to 35.3% and achieving near‑oracle performance while being significantly faster.
By Mingkai Liao, Zijun Zhang, Mingyang Zhou
The paper introduces VERA-RL, a reinforcement‑learning framework for proactive scientific error verification in academic papers. It builds on a Reason–Verify–Scan workflow and presents VERA‑13K, a 12,900‑sample dataset with 4,300 matched reasoning chains covering six error categories across natural‑science domains. The authors also define fine‑grained rewards for reasoning completeness, evidence alignment, and error precision, and show that training Qwen3‑VL‑8B with VERA‑RL improves verifiable reasoning to levels comparable with flagship multimodal large language models.
By Rongjin Li, Yuanxin Liu, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
The paper introduces Safe Contrastive Reinforcement Learning (Safe-CRL), a method that corrects bias in contrastive RL caused by failure-terminated Markov decision processes. By applying mass-weighted InfoNCE and a log-survival-mass score, Safe-CRL uses only a one-bit failure signal to improve survival and goal-reaching performance across twelve robot navigation and locomotion tasks. The approach demonstrates complex failure-avoidance behaviors and completes the theoretical foundation of contrastive RL under failure termination.
By Guopeng Li, Yiyang Duan, Yiru Jiao, Chengcheng Xu
AirLLM introduces a hierarchical diffusion policy framework that adapts Low‑Rank Adaptation (LoRA) rank configurations for remote fine‑tuning of large language models over wireless channels. The system uses a Proximal Policy Optimization agent to make coarse decisions based on wireless and linguistic cues, then refines these decisions with Denoising Diffusion Implicit Models to produce task‑ and channel‑specific rank vectors. Experiments across different signal‑to‑noise ratios show that AirLLM improves fine‑tuning performance while substantially lowering transmission costs.
By Shiyi Yang, Xiaoxue Yu, Rongpeng Li, Jianhang Zhu, Zhifeng Zhao, Honggang Zhang
The paper investigates the reliability of rule- and model-based verifiers used in reinforcement learning with verifiable reward (RLVR) for mathematical reasoning. It finds that rule-based verifiers often miss equivalent answers in different formats, causing false negatives that degrade RL performance as models improve. Model-based verifiers achieve higher static accuracy but become vulnerable to reward hacking during RL, misclassifying certain response patterns as correct after fine-tuning.
By Yuzhen Huang, Weihao Zeng, Xingshan Zeng, Qi Zhu, Junxian He
The paper introduces memoroids, a monoid‑based framework that reformulates memory models like RNNs and Transformers for better handling of long sequences in reinforcement learning. It critiques traditional batching approaches in recurrent RL, identifies their shortcomings, and proposes a new batching method using memoroids that enhances sample efficiency, boosts return, and simplifies recurrent loss implementation.
By Steven Morad, Chris Lu, Ryan Kortvelesy, Stephan Liwicki, Jakob Foerster, Amanda Prorok
The paper introduces MCCE, a hybrid framework that combines a frozen closed‑source large language model (LLM) with a lightweight, trainable model for multi‑objective discrete optimization. By maintaining a trajectory memory and refining the small model through reinforcement learning, the two models jointly enhance global exploration and learning. Experiments on drug‑design benchmarks demonstrate that MCCE achieves state‑of‑the‑art Pareto front quality, outperforming existing baselines.
By Nian Ran, Zhongzheng Li, Yue Wang, Qingsong Ran, Xiaoyuan Zhang, Shikun Feng, Richard Allmendinger, Xiaoguang Zhao
Behavior2Trip introduces a new task—Behavior‑Aware Travel Planning—where user preferences are inferred from past behavior trajectories rather than explicit instructions. The benchmark contains 11,400 instances from a major Chinese travel platform, each with nearly 40 recorded behaviors across 14 attributes and 5 preference dimensions. A reinforcement‑learning agent, B2T‑Agent, leverages these trajectories, external retrieval tools, and internal memory, outperforming GPT‑4.1 and other baselines on the dataset.
By Zihao Cheng, Yingyu Shan, Hongru Wang, Zeming Liu, Xinyi Wang, Xiangrong Zhu, Yuhang Guo, Wei Lin, Yunhong Wang
The paper introduces Private Best-of-N (PrivBoN), a method that adds calibrated Gumbel noise to reward scores during inference-time alignment, achieving both ε-differential privacy and KL-regularized alignment. When the privacy budget exceeds a critical threshold ε*, the noise becomes regret-optimal, matching the theoretical alignment skyline. The authors also propose Private Inference-Time Pessimism (PrivITP), which uses χ^2-regularized rejection sampling and a two-phase Gaussian mechanism to provide ex-post (ε,δ)-DP with a privacy cost independent of the number of responses, and demonstrate that both methods outperform standard Best-of-N across multiple models and datasets.
By Ishi Jain, Nandini Bhattad, Sayak Ray Chowdhury
The paper "Performance Foundations of Parallel & Distributed Reasoning Language Models" examines how reinforcement learning with verifiable rewards (RLVR) and similar post‑training methods improve reasoning in large language models, yet demand massive computational resources. It provides a compute‑centric analysis of key RL frameworks such as PPO and GRPO, and introduces a taxonomy of intra‑ and inter‑model parallelism strategies—including data, tensor, pipeline, sequence, context, expert, disaggregated placement, stage fusion, hybrid parallelism, and asynchronous execution—to address the parallel and distributed systems challenges of training reasoning language models. The authors also analyze existing RLM frameworks, offering practical guidelines and outlining open research directions for building scalable, fast, and cost‑effective RLMs.
By Maciej Besta, Leonard Schmidt, Lara Nonino, Robert Gerstenberger, Pierre Pang, Patrik Okanovic, Ales Kubicek, Tiancheng Chen, Baraq Lipshitz, Torsten Hoefler
AffectOmni is a reinforcement‑learning‑trained framework that enhances multimodal large language models for affective reasoning in social and art‑related scenes. It introduces People Focus and Temporal Order rewards to prioritize people‑centric cues and structured reasoning, and uses within‑group comparative scoring for more discriminative rewards. A Thinking Summarizer converts rationales into executable evidence instructions, which are grounded into pixel‑level regions via SAM3, enabling external auditability.
By Yibo Wang, Rui Yang, Jisheng Dang, Bimei Wang, Yitao Wu, Pengfei Cao, Wencan Zhang, Hong Peng, Bin Hu, Tat-Seng Chua
The paper introduces Agentic Reasoning for Tree Search (ARTS), a method that uses a reasoning language model to navigate the hypothesis‑experiment space in scientific discovery. Unlike traditional approaches that conflate hypothesis quality with execution quality and prune search logs, ARTS evaluates prior execution logs to distinguish implementation failures from poor hypotheses and selects the next hypothesis to pursue. By employing test‑time training to embed search‑tree knowledge into model weights, ARTS achieves a 15.3% relative improvement over leading algorithms on 22 benchmark tasks and enables smaller models like Qwen3‑4B to match or exceed the performance of larger closed‑source models at lower inference cost.
By Gurusha Juneja, Arnav Kumar Jain, Deepak Nathani, William Yang Wang, Xin Eric Wang
The paper introduces LAC (Light Actor, deep Critic), an offline reinforcement learning approach that allocates model capacity to a deep critic rather than a complex actor to improve inference efficiency. It addresses three failure modes—optimization, bootstrap-noise amplification, and value-range drift—using a residual MLP backbone, n‑step bootstrap targets, and a categorical cross‑entropy loss. Experiments on OGBench show LAC matches state‑of‑the‑art diffusion and flow‑matching baselines while reducing inference latency by up to four times.
By Guhyeon Kang, Jaehwi Lee, Minhae Kwon
CARE: Causally-Aligned Reasoning Exploration for Medical Large Language Models proposes a new framework to improve medical reasoning in LLMs. It introduces two key conditions—Causal Sufficiency and Proximal Learnability—to curate high-quality training trajectories, using agreement-based self-verification and dynamic entropy bounds. Experiments on medical multimodal and text-only benchmarks show that CARE outperforms competitors, reducing incorrect reasoning and enhancing training stability.
By Yucheng Zhou, Peng Luo, Qianning Wang, Chengzhong Xu, Jianbing Shen
The paper investigates whether the high costs of training chain-of-thought reasoning models can be reduced through algorithmic design. It introduces an autocurriculum approach that lets the model select which problems to focus on during training, showing that this method provably improves both supervised fine‑tuning and reinforcement learning. For supervised fine‑tuning, autocurriculum requires exponentially fewer reasoning demonstrations by targeting prompts where the model struggles, while for reinforcement learning it decouples computational cost from the quality of the reference model, making the burn‑in cost nearly independent of target accuracy.
By Nived Rajaraman, Audrey Huang, Miro Dudik, Robert Schapire, Dylan J. Foster, Akshay Krishnamurthy
The paper introduces Reward‑Informed Sparse Autoencoders (RI‑SAEs), which use reinforcement‑learning rewards to curate data for training sparse autoencoders on language‑model activations. On Llama‑3.1‑8B, a sparse subset of features separates high‑reward from low‑reward reasoning continuations, but control experiments show this separation largely reflects solution completeness rather than true reasoning quality. The authors conclude that reward filtering can cheaply reuse RL signals for interpretability, though most of the discovered features capture completion form rather than deep reasoning.
By Tanvi Nagilla, Alexander Jameson, Daniel Manta, Shayaan Uddin
Self-OPD introduces a teacher‑free on‑policy distillation framework for flow matching models, using the student’s own exploration to generate step‑wise supervision. At each timestep the deterministic next‑state prediction is branched into multiple stochastic SDE candidates, rolled out, and compared against a deterministic baseline to compute normalized advantages. The velocity field is then optimized with a pull‑push objective that attracts high‑advantage branches and repels low‑advantage ones, while multi‑objective alignment is achieved by fusing normalized scores at the reward level.
By Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He, Siyu Zou, Jinlong Liu, Yunlong Yu, Jian Song, Hao Jiang, Pipei Huang, Bo Zheng
The paper presents a modeling and simulation framework to study reinforcement learning (RL) control of connected and automated vehicle (CAV) platoon joining maneuvers in mixed traffic. Using SUMO and agent-based modeling, it evaluates Deep Q-Network (DQN), Double DQN (DDQN), and Proximal Policy Optimization (PPO) algorithms, finding that PPO achieves a 98 % joining success rate with less than 1 % collision rate by incorporating risk penalties. The study also shows a trade‑off between safety, joining effectiveness, and decision efficiency, and demonstrates that an external safety controller can prevent collisions but may reduce joining efficiency.
By Biao Yin, Abderrahmane Kasmi, Nadir Farhi
Think3D introduces a framework that endows Vision‑Language Models with interactive 3D chain‑of‑thought reasoning by integrating 3D manipulation tools for active spatial exploration. The approach improves performance on benchmarks such as BLINK Multi‑view, MindCube‑1K, and VSI‑Bench‑Tiny for proprietary models like GPT‑4.1 and Gemini 2.5 Pro, and a reinforcement‑learning variant, Think3D‑RL, enables open‑weight models such as Qwen3‑VL‑4B to autonomously learn effective 3D exploration strategies, yielding tool‑use patterns comparable to stronger models and turning a performance drop on MindCube‑1K into a substantial improvement.
By Zaibin Zhang, Yuhan Wu, Lianjie Jia, Yifan Wang, Zhongbo Zhang, Yijiang Li, Binghao Ran, Fuxi Zhang, Zhuohan Sun, Yizhuang Peng, Zhenfei Yin, Lijun Wang, Huchuan Lu