Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.28716v1 Announce Type: new Abstract: Effective decision-making in human-centric environments such as Intelligent Tutoring Systems (ITSs) and clinical healthcare requires policies that are...
arXiv:2610.29538v1 Announce Type: new Abstract: Direct preference optimization (DPO) incurs training costs in output scoring, sequence execution, and backpropagation, so reducing supervision alone do...
arXiv:2610.28818v1 Announce Type: cross Abstract: As systems engineering (SE) shifts from monolithic systems to Systems of Systems, Mission Engineering (ME) has emerged as a new line of thinking in S...
arXiv:2610.28858v1 Announce Type: cross Abstract: Distilled diffusion models generate high-quality images in a few sampling steps, but aligning them with task requirements and human preferences is di...
arXiv:2610.28925v1 Announce Type: cross Abstract: Modern robotic agents are increasingly expected to support steerable control, where a single policy can adjust its behavioral style at runtime under...
arXiv:2610.29595v1 Announce Type: cross Abstract: In this paper, we study quantile distributional policy evaluation, where the return distribution under a given policy is approximated by the quantile...
arXiv:2610.30097v1 Announce Type: cross Abstract: Vision-language-action (VLA) models have shown promise in robotic manipulation, but imitation learning (IL) alone often struggles to master complex l...
arXiv:2605.08802v4 Announce Type: replace Abstract: Due to the potential for exploratory reasoning of Latent Visual Reasoning, recent works tend to enable MLLMs (Multimodal Large Language Models) to...
arXiv:2610.29845v1 Announce Type: cross Abstract: Autonomous mobile robots operating in dense crowds and cluttered sites require $360^\circ$ perception to react to surrounding hazards. While panorami...
arXiv:2603.22364v4 Announce Type: replace Abstract: Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifi...
arXiv:2610.29410v1 Announce Type: new Abstract: Preference optimization can improve every ranking metric while building a hidden, shared representation shift that the relative loss barely sees, one t...
arXiv:2610.29671v1 Announce Type: new Abstract: Long-term agent memory is shifting from predefined workflows toward learnable policies for memory construction and retrieval. Existing learning-based m...
arXiv:2610.24899v2 Announce Type: replace-cross Abstract: While LLM agents are increasingly capable of gathering and synthesizing evidence from a single homogeneous source, such as structured tables...
arXiv:2610.28776v1 Announce Type: new Abstract: Conventional surgical segmentation relies on closed-set categories or explicit text queries, which cannot resolve implicit queries that require reasoni...
The paper introduces a continuous‑time policy gradient framework that works with discrete observation data. It derives performance difference lemmas and policy gradient formulas for randomized and parametrized policies, proving exponential convergence for an entropy‑regularized policy flow and providing explicit convergence bounds for discrete updates. By integrating with PhiBE, the authors create model‑free actor‑critic algorithms that maintain the continuous‑time structure, and demonstrate their effectiveness on both linear‑quadratic and nonlinear robotic control tasks.
The paper introduces Hybrid Belief Reinforcement Learning (HBRL), a two‑phase framework for coordinating multiple autonomous agents in spatial exploration tasks. In phase one, agents build spatial beliefs via a Log‑Gaussian Cox Process and plan information‑driven trajectories using a Pathwise Mutual Information planner; in phase two, a Soft Actor‑Critic policy is warm‑started through dual‑channel knowledge transfer that incorporates belief states and replay‑buffer demonstrations. Experiments on UAV wireless service provisioning and NYC taxi demand sensing show significant performance gains, with the dual‑channel transfer outperforming single‑channel approaches.
The paper investigates how data should be scaled in agentic reinforcement learning, where data is now a structured design space of tasks, environments, and trajectories. By fitting per‑axis scaling laws across various large language models and RL algorithms, the authors find that environment diversity has the strongest impact, followed by task and trajectory diversity. Using these laws, they price synthesis methods, identify critical synthetic‑to‑real ratios, and formulate a compute‑optimal budget allocation that can improve efficiency by up to 1.7× compared to the common practice of scaling trajectories alone.
The paper investigates how reward misspecification in reinforcement learning harms the ability to monitor chain-of-thought (CoT) reasoning in language models. By creating a testbed with true and misspecified rewards, the authors show that models can hack the reward while still producing CoTs that appear aligned, leading to low monitorability. They propose a mitigation strategy that collects diverse reward rollouts, infers a reward description with a language model, and injects this description into the input, improving monitorability by up to 42% absolute across models and tasks.
The paper introduces RoboJudge, a multimodal language model trained to evaluate embodied video generation. It presents RoboJudge-Bench, a benchmark of 800 human‑annotated videos assessing Physical Adherence and Instruction Alignment, and RoboJudge-Train, a 12K‑video dataset with human‑verified scores and rationales. Fine‑tuned on this data, RoboJudge‑9B achieves the highest agreement with human judgments, outperforming existing models and improving performance on external physics benchmarks.
The paper examines inference‑time observation attacks on Safe Multi‑Agent Reinforcement Learning (SMARL), where an adversary perturbs observations of some agents to alter their actions. It introduces the MinMax Attack framework, which jointly selects adversarial target actions for compromised agents while considering the cooperative resistance of benign agents. The authors derive a bound on the induced safety cost and show through experiments that MinMax Attack causes higher safety costs than existing baselines while maintaining task reward, thus improving attack effectiveness and stealthiness.