The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.
By Yichen Lin, Xuyuan Xiong, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao
The paper introduces TalkMesh, a decentralized network of small language model agents that learn to communicate effectively during inference. Each agent proposes an answer, scores it with a confidence head, and the most confident agent broadcasts a hint; lower‑confidence agents revise their proposals if a new suggestion scores higher. This gossip‑based consensus, trained via group relative policy optimization, enables a mesh of three agents to match the accuracy of majority voting over 32 samples, and scales to larger meshes to significantly boost performance on benchmarks like GSM8K and MATH-500.
By Mehmet Kerem Turkcan
The paper introduces a new method for generating universal adversarial perturbations (UAPs) against deep reinforcement learning (DRL)-based intrusion detection systems (IDS). It leverages Probabilistic Robustness (PR) as a post‑hoc metric to guide UAP creation, integrating PR directly into the optimization objective. The authors further develop PX‑UAP, which incorporates explainable AI (XAI) to shape perturbations within realistic domain constraints, and provide a theoretical analysis of its design. Experiments show PX‑UAP outperforms existing UAP techniques in attack effectiveness.
By Hongsen Zhang, Lu Zhang, Mingjing Xu, Yi Zhang, Gregory Epiphaniou, Carsten Maple
The paper introduces a new approach to learning chance-constrained Markov decision processes (CCMDPs) using a Bellman distributional certificate. It provides both model-based and model-free algorithms with theoretical guarantees, including matching upper and lower bounds for tabular discounted CCMDPs with bounded successor support. Numerical experiments on synthetic CCMDPs and an IEEE 14-bus energy storage benchmark demonstrate the safety and effectiveness of the proposed methods.
By Chenbei Lu, Hongyu Yi
HySTAR is a MAPPO-based framework that addresses structural target drift in cooperative multi‑agent reinforcement learning by anchoring an overlapping sparse hypergraph as a stable high‑order value‑decomposition scaffold. It separates adaptive representation learning from a temporally consistent decomposition basis, using a spatiotemporal encoder to capture physical and task‑dependent interactions and combining temporal and structural relevance to compute agent‑specific advantages. Experiments on SMAC, GRF, Traffic Junction, and MPE show consistent improvements over MAPPO‑style, value‑factorization, and dynamic‑grouping baselines, achieving significant gains in performance and convergence speed.
By Xinglong Luo, Yuding Zhang, Yuheng Kuang, Shuxuan Yuan, Zhenni Zeng, Weiqiang Zhu, Zhenhai Ji, Zhengning Wang
The paper introduces an online model‑based reinforcement learning framework that learns a probabilistic dynamics ensemble from scratch for sampling‑based model predictive control, specifically targeting precise, high‑speed control of hydraulic excavators. A precision‑gated contouring objective prioritizes path accuracy over speed, enabling the system to achieve higher sample efficiency than existing model‑based RL baselines in a data‑driven simulator. The method is validated on an 11.5‑ton Menzi Muck M445 excavator, reaching tracking accuracy comparable to prior controllers after only 20 minutes of real‑world interaction and maintaining sub‑centimeter mean path error at high speeds after 40 minutes.
By Claudio Canales, Fang Nan, Marco Hutter, Javier Ruiz-del-Solar
OpenHail is an open-source Gymnasium environment designed for controlling electric ride‑hailing fleets. It offers a fixed‑size observation–action interface that handles request assignment, repositioning, and charging, while its event‑driven simulator models pickup deadlines, vehicle job queues, battery dynamics, and finite‑capacity charging facilities with FIFO queues. The environment supports various decision‑epoch mechanisms—event‑driven, periodic, hybrid, and policy‑requested—allowing flexible policy interactions within a unified operational model, and includes tools for evaluation, metrics, and baseline policies.
By Tommaso Schettini, Nicholas D. Kullman, Jorge E. Mendoza
The paper introduces robust successor features, a method that extends the successor representation to handle uncertainty in both reward functions and transition kernels within linear Markov Decision Processes. It provides a theoretical bound on Generalized Policy Improvement that quantifies performance loss due to mismatched dynamics, and demonstrates the approach on grid-based benchmarks against prior methods that consider only reward or transition differences.
By Erik Nikulski, Yamen Habib, Vicen\c{c} Gomez, Anders Jonsson, Rub\'en Moreno-Bote, Javier Segovia-Aguas
The paper introduces Geometric Anchor Preference Optimization (GAPO), a method that replaces the static reference policy in Direct Preference Optimization with a dynamic, geometry-aware anchor—a small adversarial perturbation of the current policy. GAPO uses this anchor to adaptively reweight preference pairs based on local sensitivity, and defines an Anchor Gap that approximates worst‑case local margin degradation. Experiments show that GAPO improves robustness to noisy supervision while matching or surpassing existing LLM alignment and reasoning benchmarks.
By Youngjae Cho, Jongsuk Kim, Ji-Hoon Kim
HybridInfer is a thermal‑aware reinforcement‑learning router that selects among on‑device, edge, and cloud large language model tiers based on a phone’s thermal headroom and query complexity. Trained offline with a Q‑learning policy, it balances quality, latency, cost, and a thermal penalty, and includes a locality bonus that encourages on‑device execution. In real Android tests on 210 prompts, the learned router outperformed two hand‑tuned heuristics in quality while maintaining the lowest cost, and it proved more reliable and faster than always‑on‑device inference, especially for long queries.
By Simran Koul
Historical street-view imagery records urban evolution, but uneven coverage leaves substantial gaps in historical records. Generating plausible past appearances requires restoring changed structures w...
The paper explores Retrospection-Only Fine-Tuning (ROFT), a method where a language-model agent improves its behavior by generating and training on explanations of its own experiences, without external teachers or reward signals. In software‑engineering tasks with Qwen3.5‑4B, ROFT achieves comparable or better solve rates than GRPO while requiring fewer updates and training time, and can learn from failures alone. Behavioral analysis shows ROFT indirectly assigns credit to actions and can produce shorter, more direct solutions when prompted to focus on direct solutions.
The paper introduces a dynamic bipartite matching framework that uses large language model (LLM) agents and contextual bandits to model decentralized, asynchronous matching processes without requiring full preference rankings. In a simulated Chinese marriage market, LLM agents evaluate local candidates while Logistic-UCB models learn reciprocal acceptance, leading to higher mutual welfare and fewer blocking pairs compared to classical Gale–Shapley. The study validates LLM-generated preferences against empirical data and demonstrates gender-differentiated acceptance patterns, supporting the use of decentralized LLM-based matching for economic simulation and computational social science.
RGDT-Bench is a new benchmark that evaluates large language models on Rule‑Governed Decision Tasks, where models must apply external rules to facts, justify decisions, and provide checkable justifications. The benchmark offers 202.1K condition‑level supervision slots across four task tracks and eight task‑probe combinations, and it labels warrant completeness through label‑blind extraction and deterministic checks. Evaluation shows that among correct responses, 40.2% of warrants are incomplete, and existing evaluators struggle to detect this, prompting the authors to train a reward model that improves AUROC to 69.24% and outperforms outcome‑supervised baselines.
The paper introduces Q-learning Penalized Transformer (QPT), a training–inference consistent framework for safe offline reinforcement learning. QPT trains a Transformer policy that generates actions conditioned on trajectory context and target return/cost while incorporating a Q-shaped penalty to balance safety, reward maximization, and behavior regularization. The method consistently outperforms strong baselines on 38 DSRL benchmark tasks and adapts robustly to varying constraint thresholds.
Dr.Credit introduces a rubric‑grounded credit assignment method that evaluates intermediate tool turns in deep research agents by comparing the information returned to the history of accepted support for each rubric. Unlike traditional approaches that rely on ground‑truth answers, Dr.Credit uses task requirements as a shared reference, distinguishing new support from previously seen evidence and recognizing partial rubric fulfillment. Experiments on four benchmarks show that Dr.Credit outperforms open deep research baselines across all primary metrics, achieving performance competitive with proprietary models while enabling more efficient evidence acquisition and higher‑quality reports under limited turn budgets.
The paper introduces CamVLM, a framework that equips large vision‑language models with the ability to actively control camera viewpoints for improved surveillance video understanding. It presents two new datasets: CCTV‑Anomaly, a large‑scale surveillance video collection with detailed captions and event annotations, and CamTrack‑53K, an object‑centric viewpoint trajectory dataset for learning camera actions. Using reinforcement learning, CamVLM learns long‑horizon observation strategies, achieving state‑of‑the‑art performance in both passive and dynamic viewpoint settings.
By Xiao Zhang, Wang Zeng, Sheng Jin, Wentao Liu, Chen Qian, Shichao Kan
Self-Play Search Distillation (SPSD) is a framework that generates superhuman synthetic data by having MuZero-like networks play board games in executable environments. The search records are converted into structured reasoning chains that serve as environment‑grounded supervision for training large language models. When applied to Qwen3‑4B‑Base, SPSD improves performance on six mathematics benchmarks from 24.1 to 36.6 and raises the win rate on unseen games from 15% to 45%.
By Lorenzo Molfetta, Wai-Chung Kwan, Giacomo Frisoni, Luca Ragazzi, Gianluca Moro, Pavlos Vougiouklis, Jeff Z. Pan, Pasquale Minervini
The paper introduces MA-WAM, a test‑time planning framework that uses a frozen multi‑agent flow policy to evaluate future joint actions by predicting their consequences while accounting for cross‑agent dependencies. Unlike naive extensions of single‑agent world models, MA‑WAM captures the interactions among simultaneous actions, enabling efficient candidate scoring. Experiments on 30 MARL benchmarks (MAMuJoCo, SMAC, MPE) show MA‑WAM improves performance by 22.0% over direct execution and 25.6% over uniform action selection, with only a 12.1 ms overhead on an A100 GPU.
By Guowei Zou, Haitao Wang, Guoxin Wang, Beiwen Zhang, Zhiquan Chen, Guojie Wang, Hejun Wu
SLMFix is a code‑generation pipeline that uses a small language model fine‑tuned with reinforcement learning to correct syntactic errors in programs produced by large language models for domain‑specific languages. The approach relies on interpreter feedback to guide the error‑fixing process. Experiments show that SLMFix improves validator pass rates by 40% on low‑resource programming languages and removes over 50% of syntactic errors on high‑resource DSLs, outperforming supervised fine‑tuning even for 7B models.
By David Jiahao Fu, Aryan Gupta, Aaron Councilman, Yu-Xiong Wang, Vikram Adve