The paper critiques the common reinforcement‑learning approach of sampling tool subsets when the full set of tools is enumerable, showing that sampling leads to degraded policy estimates and increased reward sparsity in genomic reasoning tasks. It proposes Full‑Group Policy Optimization (FGPO), which evaluates every tool subset and precomputes rewards in a table, thereby eliminating the need for frozen‑reasoner calls during training. Experiments across five frozen reasoners and three genomic benchmarks demonstrate that FGPO consistently outperforms GRPO, improving average scores by 6.75 points and reducing the number of invoked tools per question.
By Haoyue Liu, Xiaoyu Ma, Ye Chen, Zhichao Wang, Xiaoying Tang
HiRAD is a hierarchical reinforcement learning framework designed for continuous-space routing of large-scale AGV fleets, offering real-time guarantees. It introduces a step-level spatiotemporal representation, separates heading selection from velocity control to shrink the action space, and employs an asynchronous event-driven decision pipeline that reduces inference complexity from O(n²) to O(n) and cuts per-step latency by up to 71%. Experiments on random graphs and two warehouse maps show that HiRAD decreases makespan by 45% to 63% and shortens overall runtime.
By Yunjie Huang, Ruizhong Wu, Mengxuan Zhang, Frodo Kin Sun Chan, Yan Nei Law, Lei Li
The paper introduces Graph-Guided Quasimetric Dense Reward (G2QDR), a framework that learns a state connectivity model to predict pairwise connectivity strengths in asymmetric environments. These strengths are converted into scalar auxiliary dense rewards, offering continuous guidance across hierarchical levels. G2QDR can be integrated into any existing Goal-Conditioned Hierarchical Reinforcement Learning architecture and shows empirical performance improvements in sparse reward settings with modest computational cost.
By Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang, Doina Precup
The paper introduces GenQAS, a tensor network‑guided reinforcement learning framework that uses a learned local transition model to generate synthetic circuit transitions for prioritized generative replay. By mixing these synthetic transitions with real experience during Double Deep Q‑Network updates, GenQAS addresses sample starvation in quantum architecture search. Across benchmarks ranging from 6 to 15 qubits, the method improves success probabilities, identifies compact circuits, and reduces steps to chemical accuracy by up to 92.7%.
By Akash Kundu, Amit Kumar Jaiswal, Sebastian Feld, Prayag Tiwari
The paper presents a method where self‑evolving scientific agents design explicit, neural‑network‑free white‑box controllers for fluid dynamics tasks. By iteratively interpreting simulation data, the agents accumulate control knowledge and refine controller code, ultimately achieving robust control of an underactuated two‑joint swimmer in unsteady flows. The resulting controllers generalize across varying target positions, wake geometries, cylinder counts, and inflow speeds, and 2D control priors successfully transfer to accelerate 3D adaptation.
By Boai Sun, Wenjin Guo, Zongmin Yu, Liu Yang
MOSAIC is an open‑source platform that allows agents from different decision‑making paradigms—such as reinforcement learning policies, large language models, vision‑language models, and human operators—to operate together in shared reinforcement learning environments. It achieves this through an IPC‑based worker protocol that isolates each agent’s training and inference logic, an operator abstraction that maps any agent to a minimal universal interface, and a deterministic evaluation framework offering both manual lock‑step and automated script modes for reproducible experiments.
By Abdulhamid M. Mousa, Jinhui Pang, Rakhmonberdi Khajiev, Jalaledin M. Azzabi, Abdulkarim M. Mousa, Peng Yong, Yunusa Haruna, Ming Liu
The paper introduces the Belief-State Engine (BSE), an inference module that supplies a large language model (LLM) with a Bayesian posterior over hidden states in a partially observable Markov decision process (POMDP). By keeping the raw action‑observation log hidden from the LLM, the BSE ensures the agent behaves as a sound Markov policy on the belief MDP, thereby inheriting classical POMDP optimality guarantees. Experiments on the Tiger POMDP and a red‑team attack‑graph task show that BSE‑augmented agents outperform six baselines in task return, belief calibration, and decision consistency.
By Arnab Chattopadhayay, Debdipta Halder
Quantum Tiq‑Taq‑Toe is a popular benchmark for quantum computing and machine learning, yet no reinforcement learning (RL) methods have been applied to it. The paper introduces RL techniques for this game, which is simpler than Quantum Chess but still challenging due to partial observability and exponential state complexity. States are represented by a 3×3 measurement matrix and a 9×9 move‑history matrix of entanglement relations, making strategy development difficult because each move can collapse the quantum state.
By Catalin-Viorel Dinu, Thomas Moerland
The paper presents an algorithm that lets a learning agent ask for help from a mentor and transfer knowledge between similar states, enabling safe and effective learning in Markov decision processes with irreversible dynamics and infinite state spaces. It proves that both regret and the number of mentor queries grow sublinearly over time, using a sequence of three reductions to achieve a general result. The work claims to be the first formal proof that an agent can achieve high reward while becoming self‑sufficient in an unknown, unbounded, high‑stakes environment without resets.
By Benjamin Plaut, Juan Li\'evano-Karim, Hanlin Zhu, Stuart Russell
The paper presents a gamified 20Q-style recommender for cybersecurity education that uses reinforcement learning and explainable AI to guide learners through interactive questioning. By acting as a knowledgeable questioner, the system narrows down user-described security scenarios, identifies the underlying threat, and transparently explains its reasoning. The authors detail the system architecture, algorithmic foundations, and provide case studies covering attack vectors such as the Cyber Kill Chain, phishing, ransomware, and web application vulnerabilities.
By Mary Nusrat, Sarfuddin Bhuiyan, Gahangir Hossain
The paper introduces topological necessities—mechanism‑invariant subgoals derived from the topology of successful trajectories—used to guide long‑horizon goal‑conditioned reinforcement learning. By computing homology in dimensions 0 and 1 over a transport‑weighted carrier, the authors obtain an enumerable gate set that forms a recursive topological gate hierarchy. These certified gates transfer across different embodiments (e.g., from PointMaze to Ant and Humanoid) without retraining, achieving state‑of‑the‑art performance on several benchmark tasks.
By Hao Shi, Xi Li
The paper introduces a Bellman optimality equation for optimizing plasticity in Markov decision processes, extending recent work that reframes the stability‑plasticity dilemma as an empowerment‑plasticity tradeoff. It builds on Abel et al. (2025), which defined plasticity as the generalized directed information from observations to actions and empowerment as the reverse. This work is the first to address plasticity optimization under that new definition, providing a theoretical foundation similar to existing empowerment‑based approaches.
By Jeremy Lucas, Doina Precup
The paper proposes a new technique that averages the logits of a frozen reference policy (such as one obtained via supervised fine‑tuning) and a trainable policy, integrating this approach into Group Relative Policy Optimization (GRPO). Unlike methods that use Kullback–Leibler regularization or a critic, the proposed method couples the policies solely through logit averaging, aiming to combine the reasoning strengths of the trainable policy with the formatting benefits of the reference policy. Experiments on MATH, cn‑k12, and MMLU demonstrate that this approach achieves higher or comparable accuracy to the standard KL‑regularized GRPO.
By Xingwei Gan, Ying Zhu
The paper introduces TRACE, a digital‑advertising diagnostic environment that uses simulated interventions to generate verifiable rewards for training reasoning agents. By injecting controlled interventions into a simulator, the hidden cause of anomalies becomes an oracle label, enabling agents to learn to identify root causes and affected segments through noisy, confounded evidence. Experiments show that reinforcement learning with these synthesized rewards outperforms large prompted baselines, achieving higher accuracy while using fewer tool calls.
By Rui Sun, Zhan Shi, Bing He
The paper investigates a natural gradient method based on the Fisher information matrix of state-action distributions, which follows a Fisher‑Rao gradient flow within the state-action polytope under a linear potential. It establishes linear convergence rates for Fisher‑Rao gradient flows of linear programs, with the rate tied to the program’s geometry, and provides improved error bounds for entropic regularization. Additionally, the authors extend their analysis to perturbed flows, proving sublinear convergence for both perturbed Fisher‑Rao and natural gradient flows, thereby encompassing state‑action natural policy gradients.
By Johannes M\"uller, Semih \c{C}ayc{\i}, Guido Mont\'ufar
arXiv:2609. 10866v1 Announce Type: new Abstract: Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications.
By Tong Li, Saunak Kumar Panda, Yisha Xiang
The paper surveys how large language models (LLMs) are being applied in legal tasks such as judgement prediction, document analysis, and drafting. It reviews the benefits of automation while highlighting legal challenges like privacy, bias, and explainability. The authors also discuss data resources for legal domain specialization and outline future research directions.
By Zhongxiang Sun
The paper presents the KuaiRP series of role‑playing models, detailing a multi‑stage training pipeline that balances deep domain knowledge injection with the preservation of general agent capabilities. The approach includes a standardized character template, a supervised fine‑tuning (SFT) data pipeline, a rule‑based reward function for reinforcement learning, and a novel two‑stage on‑policy distillation (OPD) with cumulative‑divergence decay (CDD) to recover general skills. Experimental results show that the models achieve state‑of‑the‑art role‑playing fidelity in target domains while maintaining low deployment costs.
By Yipeng Wang, Ziwei Zhang, Jiahui Zhang, Qi Gan, Kai Sheng
The paper introduces the concept of proof‑carrying cognition, aiming to close the verification gap in language‑model reasoning by using reality‑settled rewards. It presents a theoretical framework linking verifier‑gold correlation to compute‑capability trade‑offs, demonstrates that unsound verifiers degrade under best‑of‑N selection while sound verifiers improve, and proposes a new benchmark metric, Soundness‑under‑Pressure, for evaluating reality‑settled reasoning systems.
By Eshwar Reddy M, Sourav Karmakar
MERIT is a two‑stage framework for reviewer assignment that first trains a reviewer assessor using reinforcement learning to match paper‑specific expertise rubrics with reviewers’ prior work, guided by an LLM judge. The assessor’s predictions are then distilled into an embedding‑based retriever for efficient large‑scale assignment. Experiments show the 4B assessor outperforms larger general‑purpose LLMs on suitability classification, and the retriever achieves state‑of‑the‑art performance on LR‑Bench and the CMU Gold dataset.
By Zixuan Yang, Yibo Zhao, Weicong Liu, Xiang Li