CM-DPO introduces a Constraint-Margin Direct Preference Optimization method that replaces the binary preference signal of traditional DPO with a continuous margin based on a deterministic symbolic verifier, scaled by violation severity. Hard and soft constraints are separated via a lexicographic objective to prevent hard constraints from being traded off against preferences. Using procedurally generated constraint profiles and minimal-edit distillation within the SynPlan-R framework, an 8B model fine‑tuned with CM‑DPO achieves high pass and solve rates on several planning benchmarks, matching multi‑agent systems with lower latency and outperforming GPT‑4o on unseen tasks.
By Rabimba Karanjai, Qun Gu, Hemanth Hegadehalli Madhavarao, Wenhuan Sun, Xiaojiao Yu, Suryabhan Singh Hada, Libin N. George, Uma Kona, Richard Williamson, Linsey Pang, Prakhar Mehrotra
The paper introduces Ask the Expert, a training-time framework that leverages large language models (LLMs) to guide reinforcement learning for autonomous cyber defense. It summarizes complex cyber‑defense states, intermittently queries an LLM for host‑level defensive recommendations via a constrained action interface, and then uses those recommendations to shape rewards for PPO training. After training, the LLM is discarded, leaving a pure RL policy that improves sample efficiency and outperforms potential‑based reward shaping baselines on TTCP CAGE CC1 and CC2 scenarios.
By Fernando Martinez, Abhishek Satyam, Tao Li, Junaid Farooq, Ying Wang, Juntao Chen
The paper investigates safe reinforcement learning by focusing on the episodic-cost tail rather than just the mean cost. It introduces the use of γαρα_{0.1} (the average cost of the worst 10% of episodes) to classify policies as tail-safe. The authors evaluate standard algorithms on Safety-Gymnasium tasks, identify tail-unsafe policies, and explore constraint families to control tail violations while maintaining return.
By Samuel Tetteh, Cody Fleming
arXiv:2610.09804v1 Announce Type: new
Abstract: Reinforcement learning is now central to eliciting reasoning in large language models, while in the popular algorithm Group Relative Policy Optimizatio...
By Yingxiang Yang, Weihang Xiao, Zhunxuan Wang, Joshua Flashner, Niresh Agarwal
The paper introduces DIGHT, a co‑adaptive framework that couples a Digital Human Interaction Generator with a Humanoid Tracking policy. DIGHT first runs multiple text‑conditioned interaction candidates in simulation, then builds physics‑grounded preferences from the rollouts—covering executability and interaction fidelity—and uses diffusion‑based direct preference optimization to align the generator without back‑propagating through the simulator. The aligned generator supplies reference motions for fine‑tuning the tracker, leading to more physically plausible and faithful humanoid interactions in simulation.
By Kerui Chen, Jianrong Zhang, Kai Lv, Hehe Fan
arXiv:2610.08963v1 Announce Type: new
Abstract: Asynchronous reinforcement learning (RL) for large language model (LLM) agents trains one policy on trajectories generated by another: rollouts come fr...
By Yifan Zhang
arXiv:2610.09226v1 Announce Type: cross
Abstract: Data collection using Unmanned Aerial Vehicles (UAVs) is challenging when LoRa IoT Devices (IoTDs) duty-cycle to conserve battery. Under energy const...
By Naresh Babu Kakarla, V. Mahendran
LLM agents increasingly improve at inference time by accumulating experience in context rather than by updating parameters. This process is often described as in-context reinforcement learning (ICRL)....
Text-to-image diffusion models are increasingly distilled into few-step variants and being deployed to enable fast inference. However, their ability to generate harmful or undesired content poses sign...
Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Existing harness-model co-evolution approaches improve...
Interpretability offers a solution to safe autonomy by providing transparency into an agent's underlying decision-making model. Within sequential-decision making tasks, differentiable decision trees (...
Agent Lightning v1.0 is a lightweight 3,500‑line framework designed to connect existing AI agents to reinforcement learning training. It simplifies the process of improving agents by handling tools, context, and decision‑making within a single, easy‑to‑integrate system. The framework allows developers to train agents with real harnesses without the need to rebuild complex underlying structures.
By Zhiyuan He, Yuqing Yang
Reality may establish that an outcome occurred without identifying which evolving procedure produced it or why. This distinction matters in production ML systems whose code, configuration, and artifac...
Model organisms of alignment-relevant behaviors (e.g., backdoors, sycophancy, spurious correlations) have emerged as a key tool for evaluating whitebox interpretability techniques. We argue that the p...
Long video understanding increasingly relies on external memory to organize massive visual streams into compact representations. However, most memory-based methods dynamically adapt how information is...
The article titled "Introduction to Reinforcement Learning: Multi-Armed Bandit Simulation in Python" explains how machines can learn independently. It focuses on a practical simulation of the multi-armed bandit problem using Python. The post was published on the Towards Data Science platform.
By Carolina Bento
Sample complexity is a widely used metric in sequential decision-making problems, defined as the number of suboptimal decisions during the interaction between the agent and an environment. We study th...
arXiv:2610.08319v1 Announce Type: new
Abstract: In proof assistants such as Lean, a generated proof must pass machine compilation checks, so evaluation needs no human scoring. Direct generation fails...
By Hanchao Zhou, Jialei Li
The paper introduces a budgeted adaptive neural‑operator framework for solving time‑dependent PDEs, combining a global Fourier neural operator with a local patch‑wise residual corrector and a set‑aware selector that decides where to refine. A macro policy governs the timing and amount of refinement, and the authors propose rollout‑verified policy improvement (RV‑PI) to evaluate feasible refinement counts via actual rollouts, converting long‑horizon advantages into conservative policy targets and accepting updates only when held‑out trajectory error improves. Experiments on shallow‑water and Brusselator benchmarks show that RV‑PI outperforms immediate‑only and random macro policies, demonstrating that the value of a local correction depends on its downstream effect rather than just its immediate error reduction.
By Ange Tong
arXiv:2610. 06950v1 Announce Type: cross Abstract: Injecting skills into a frozen language model currently costs a million parameters and a reinforcement-learning pipeline.
By Ran Li, Lei Chen