The paper introduces a dataset‑free approach to generate planar graph embeddings that meet strict structural and geometric constraints. By integrating parametric graph grammars with safe reinforcement learning, the authors formulate the task as a constrained Markov decision process and use action projection to keep actions within safe sets. Their method consistently outperforms classical and deep generative baselines on a new benchmark suite for constrained, goal‑directed planar graph generation.
The paper introduces a reinforcement learning framework for optimizing workload‑aware power delivery networks (PDNs) in VLSI chips. By mapping system‑level power traces to spatial power density distributions, the method adaptively allocates PDN resources and uses a Deep Q‑Network to perform wire‑width optimization, reducing PDN area while meeting electromigration and IR‑drop constraints. Experiments on multiprocessor floorplans with PARSEC and SPLASH‑2 workloads show a 47% area reduction and a 26× speedup over simulated annealing.
The paper introduces ARRO, a reinforcement‑learning framework that enforces minimal‑change editing by auditing source images, instructions, and edited outputs for unimplemented and unintended changes. Using a vision‑language reward model and a group‑level rubric, ARRO achieves higher EditScore on multiple benchmarks and reduces off‑target pixel changes by 8.4% on FLUX.1 Kontext‑dev. The approach eliminates the need for per‑instruction human annotations and demonstrates strong performance across several evaluation sets.
ThunderSyncRL is a training framework that eliminates idle time in agentic reinforcement learning by starting gradient computation immediately once all necessary inputs are available, thereby avoiding policy staleness. It applies to group relative policy optimization (GRPO) by computing trajectory score gradients as soon as rewards arrive, and to on‑policy distillation (OPD) by updating gradients for completed agentic turns while tool calls execute. Experiments on SWE‑bench Verified and Terminal Bench 4.0 show that ThunderSyncRL matches synchronous training performance up to 1.9× faster and outperforms asynchronous training by up to 2.47 percentage points at a fixed budget.
The paper demonstrates that off‑policy merging, called grafting, outperforms on‑policy self‑distillation for continual learning. Grafting learns updates on an earlier donor checkpoint, scales the update, and optionally masks sensitive directions, thereby reducing interference with existing capabilities. Across various continual learning scenarios, grafting achieves better new‑task and old‑task performance without costly on‑policy sampling.
UniIntervene++ is an adaptive intervention agent that learns to balance autonomous execution and assisted behaviors during online reinforcement learning. It models the evolving policy, trajectory correction, and task-structured CodePolicy as Options in a unified semi‑Markov decision process, learning their values online. The agent periodically probes the policy to keep control allocation responsive, and uses coupled experience learning to improve both the policy and future interventions, achieving an 89.67% success rate on five real‑world tasks while cutting human intervention by 94.6%.
By Yudong Lin, Haoyuan Deng, Zhuoxuan Yuan, Zaijia Yang, Yuanjiang Xue, Ziwei Wang
AdaStep introduces an adaptive step-credit weighting technique for agentic reinforcement learning, addressing the coarse granularity of trajectory-level objectives in long-horizon LLM agents. By formulating the weighting as a mean-squared-error estimation problem and deriving an optimal per-state shrinkage coefficient, AdaStep selectively preserves local credit when return variation is due to the chosen action and suppresses it when downstream randomness dominates. The method requires only lightweight scalar computations, no critic or extra rollouts, and demonstrates consistent performance gains across three model backbones on ALFWorld, WebShop, and ScienceWorld.
By Xin Wang, Wenhao Wu, Menghao Zhang, Zhi Wang, Kun Shao, Jian Luan
Wayfarer is a domain‑agnostic, online deep RL agent that discovers options via Laplacian representation learning from high‑dimensional observations and uses them for control. The discovered options improve exploration, accelerate credit assignment, and generalise to unseen settings, leading to faster learning of complex policies. Wayfarer achieves state‑of‑the‑art performance among single‑stream agents on the most challenging Atari 2600 games, especially those requiring long‑horizon exploration such as Montezuma's Revenge and Private Eye.
By Erik M. Lintunen, Marlos C. Machado
Pivot‑SD is an offline self‑distillation framework for masked diffusion language models that focuses training on high‑impact commitments, called pivots, identified by an information‑gain metric. By supervising only these pivots—using cross‑entropy for successful trajectories and targeted unlikelihood for failed ones—Pivot‑SD improves LLaDA‑8B‑Instruct on math and code benchmarks with just 200 questions and four rollouts each.
By Seo Hyun Kim, Sunwoo Hong, Younwoo Choi, Chen-Hao Chao, Se-Young Yun, Rahul G. Krishnan
The paper introduces a new loss function called the horizon loss for training classifiers. It argues that the exact policy gradient used in reinforcement learning is myopic, whereas cross‑entropy is patient, and the horizon loss interpolates between the two by truncating the total error at the remaining learning. Experiments on MNIST and ImageNet with ResNet and ViT models show that horizon loss consistently improves top‑1 accuracy over cross‑entropy, especially when label noise is present.
By Ian Osband
The paper introduces Adaptive Mask Selection and Composition (AMSC), a method that estimates task similarity using non‑parametric Wasserstein embeddings from state‑action‑reward samples. AMSC selects and weights previously learned policies based on z‑score‑normalized sparsemax similarity scores, forming a prior for new tasks. Experiments on CT‑graph, MiniGrid, and Continual World show that AMSC improves mean performance and forward transfer while avoiding forgetting, highlighting the importance of selecting relevant prior knowledge and its layer‑specific composition.
By Saptarshi Nath, Inish M. D'Souza, Antonio Carta, Soheil Kolouri, Andrea Soltoggio
The paper introduces VIGOR, a framework for zero‑shot visual generalization in model‑based reinforcement learning. VIGOR enforces latent‑space consistency through asymmetric weak‑to‑strong augmentations, dynamics‑level consistency, and encoder‑level stabilization, allowing the agent to handle unseen visual distractions while maintaining sample efficiency. Experiments on the DeepMind Control Suite and Robosuite demonstrate that VIGOR outperforms state‑of‑the‑art baselines, achieving significant gains in both environments.
By Mingyu Park, Samyeul Noh, Hyun Myung, Donghwan Lee
The paper introduces Turnover-Orthogonal Credit Assignment (TOCA), a method for open-team multi-agent reinforcement learning that separates the effects of agent actions, pure turnover, and action–turnover interactions in the team return. TOCA uses a permutation‑invariant centralized critic with event tokens to provide counterfactual per‑agent credit signals and a softly weighted interaction variant, TOCA‑β, for high‑variance environments. Experiments show that TOCA outperforms event‑aware MAPPO critics and that removing interaction credit degrades performance, especially in high‑turnover settings such as the Dynamic Spread benchmark.
By Amit Thakur, Mukesh Singhal
The paper introduces Bidirectional Voronoi-biased Exploration Curriculum (BVER), a method that expands start states from the goal and goals from the initial state simultaneously, guiding both toward each other to train a single goal-conditioned policy. Inspired by bidirectional RRT planning, BVER biases exploration toward unexplored task space and steers the two expansions together. Experiments on point-mass mazes, quadrupedal box climbing, and robot-arm ring-on-peg transfer show that BVER learns faster than other reference-free curricula, achieving high success rates and robustness without requiring demonstrations.
By Juri Pfammatter, Kaixian Qu, Clemens Schwarke, Victor Klemm, Marco Hutter
NeutronGym is the first executable environment that lets language‑model agents design neutron instruments, using tools that validate their builds, McStas ray‑tracing for simulation, and a level‑resolved grading ladder that evaluates syntax, runtime, structure, and science without an LLM judge. The platform provides procedural families of instrument layouts with held‑out parameter regimes and a curated set of 16 tasks from published instruments, called McStasBench, which includes memorization probes and a sandbox. Experiments show that while seven models can reproduce at most seven of the 16 tasks and none reaches a reference design, reinforcement learning can dramatically improve performance—e.g., Qwen3‑8B’s success rate jumps from 11% to 77% on held‑out instances—highlighting the ladder’s importance for partial credit and the potential of RL to match classical optimizers under realistic simulation budgets.
By Lijie Ding, Changwoo Do
The paper introduces a hierarchical model-based reinforcement learning framework that learns control policies from action-free time series by exploiting shared dynamics across related systems. It uses low-dimensional dynamical embeddings to capture both shared structure and individual variation, which then parameterize shared policy and value networks. Experiments on Lorenz-63, double-pendulum, and neural-behavioral data show that these hierarchical policies outperform independently trained ones, achieve higher rewards than planning with the same models, and generalize to unseen systems using only embedding inference.
By Niklas Emonds, Georgia Koppe
The paper investigates whether a proximal policy optimisation (PPO) agent can learn to act as a broker in a continuous‑time broker‑trader game that has an analytically solved optimal strategy. By discretising the continuous‑time reward and validating the implementation, the authors show that a PPO agent with a feed‑forward neural network can approximate the reference action when there is no uninformed order flow, but struggles under stochastic uninformed flow, with critics failing to rank actions reliably. The study further demonstrates that a causal certainty‑equivalent controller based on observable history outperforms PPO under partial information, and that freezing the analytical policy and fine‑tuning PPO after a change in execution cost yields a measurable improvement.
whyItMatters:"The analytical solution serves both as a diagnostic benchmark for RL performance and as a practical starting policy that can be adapted to changing market conditions, illustrating how theoretical finance models can guide and improve reinforcement learning in complex financial control tasks."
By Siu Tung Wong (Institute of Finance and Technology, University College London), Carlo Campajola (Institute of Finance and Technology, University College London, UZH Blockchain Center)
HARPO is a reinforcement learning framework that jointly optimizes faithfulness and creativity in language generation. It uses a Hallucination-Aware Generative Reward Model (HA‑GRM) to evaluate both faithfulness and writing quality, and a Selective Activation Mechanism (SAM) that applies writing rewards only to hallucination‑free outputs. Experiments on Qwen models show that HARPO improves faithfulness scores and reduces hallucination rates while boosting creative‑writing performance.
By Tiezheng Yu, Yuxin Jiang, Jinpeng Li, Shuning Sun, Fei Mi, Haoli Bai, Lifeng Shang
The paper introduces ECHO-$k$, a self-supervised, task-agnostic method for selecting which modalities to acquire at test time in multimodal, high-dimensional learning. By using a deep model’s pretrained representations as proxy targets, ECHO-$k$ learns a reinforcement‑learning policy that sequentially chooses informative modalities, providing theoretical guarantees in a linear setting. Experiments show that ECHO-$k$ consistently improves budgeted downstream performance across various foundation‑model backends, offering a principled approach to cost‑aware test‑time deployment when measurements are expensive or time‑constrained.
By Eeshaan Jain, Linus Bleistein, Bart Deplancke, Charlotte Bunne
The paper introduces AdsCVR, a benchmark for e‑commerce cross‑video reasoning with 2,483 videos and 6,110 QA pairs across six reasoning dimensions. It proposes AdSeek, an agentic framework that actively selects visual and audio tools during multi‑turn exploration and uses an offline trajectory rectification mechanism to improve reinforcement learning. AdSeek achieves 74.30% accuracy on AdsCVR, outperforming its backbone by 27.90 percentage points and generalizes to the open‑domain CrossVid benchmark.
By Jinghan Zhao, Yiman Hu, Liang Wu, Jian Xu, Bo Zheng