Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2608.13888v2 Announce Type: replace Abstract: Fashion Outfit Composition (FOC) requires sequentially assembling fashion items into a stylistically cohesive ensemble. Existing works struggle to...
arXiv:2609.34851v2 Announce Type: replace Abstract: Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprec...
arXiv:2503.10118v3 Announce Type: replace-cross Abstract: The sim-to-real gap remains a critical challenge in robotics, hindering the deployment of algorithms trained in simulation to real-world syst...
arXiv:2602.23934v2 Announce Type: replace-cross Abstract: This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectu...
arXiv:2610.01291v1 Announce Type: new Abstract: Recent advances in deep learning for shadow removal have significantly enhanced image quality and realism. However, most approaches rely on real-world...
arXiv:2610.01973v1 Announce Type: new Abstract: Reinforcement learning (RL) for video generation usually assigns one scalar reward to an entire sampled video. Yet a video is not uniformly flawed: som...
arXiv:2610.02181v1 Announce Type: new Abstract: We present OmniSeek, an agentic framework that transforms an Omni Large Language Model (Omni-LLM) into an active, multi-turn reasoning agent with nativ...
arXiv:2610.00360v1 Announce Type: cross Abstract: Reinforcement learning (RL) for dexterous manipulation must discover finger-object contacts and then control the object precisely; the action noise t...
The paper introduces PCSR-Bench, a benchmark of 84,373 question‑answer pairs derived from 2,600 omnidirectional images across 26 indoor environments, designed to evaluate perspective‑conditioned spatial reasoning (PCSR) in multimodal large language models (MLLMs). It reports a significant perception–reasoning gap, with accuracy dropping from 57.59% on limited field‑of‑view reasoning to as low as 0.64% on open‑ended compositional directional chains. An RL‑based diagnostic study on a 7B‑scale model shows that reward shaping can improve performance to 60.06% on a controlled task, indicating partial plasticity of PCSR capabilities.
RISED introduces a framework that uses rubric-based textual feedback to improve training of a single large language model (LLM) agent across multiple interactive environments. By having an LLM judge tag rollouts with a shared rubric vocabulary, the system guides both online data selection and policy supervision, enabling richer cross‑environment relationships and within‑group reward contrast. Experiments show that RISED achieves the highest mean pass rate and ranks first or second in every individual environment, with rubric analysis revealing behavioural changes behind these gains.
The paper introduces the Model-Corrected World Model (MC‑WM), a method for model‑based reinforcement learning that mitigates simulator‑to‑target shift by partitioning initial target data into fit, selection, and calibration sets and choosing the family with the lowest standardized calibration risk. It employs a learned confidence signal and deterministic validity predicates to weight one‑step imagined policy updates, avoiding the need to rewrite physical rewards. The approach is evaluated on 540 unique run cells across three controlled MuJoCo dynamics with contact shifts, with one exact‑routing cell repeated after a pre‑deployment artifact gate, totaling 541 completed executions.
The paper introduces Dependency‑Aware Reward Shaping (DARS), a method that assigns step‑level credit in reinforcement learning by modeling task progress as a graph of predicates with prerequisite relations. Annotators mark each step’s effect on predicates, and DARS discounts verified predicates based on distance from broken prerequisites while preserving independent ones, converting these annotations into signed per‑step rewards. Experiments on five task families with models ranging from 1.5B to 8B show that DARS improves success rates by up to 10 points over GiGPO, boosts WebShop and Search‑R1 QA scores, complements AEPO on AIME24/25, and outperforms OmniOPD in tool‑free reasoning, with ablations confirming the contribution of step‑level credit, dependency attenuation, and graph topology.
PPO‑HRAP introduces a hybrid regime‑aware policy that blends Proximal Policy Optimization with a volatility‑conditioned regime prior to balance upside participation and drawdown control in trading. The agent uses market and portfolio features, rewards that combine log return, VIX‑conditioned drawdown penalty, exposure deviation, and turnover cost, and outputs a blended action between the PPO actor and the regime‑derived target exposure. In backtests on SPY (2020‑2022) it achieved a 27.62% total return, 8.48% annualized return, and reduced maximum drawdown from 34.10% to 18.47%, while maintaining stable performance across multiple seeds and ranking first on total return and Sharpe ratio in single‑run cross‑asset tests on QQQ and DIA.
arXiv:2610.00188v1 Announce Type: cross Abstract: Voxel-based volumetric mapping is fundamental to 3D reconstruction, yet fixed-resolution grids remain inherently inefficient - wasting memory in unif...
The paper introduces a new approach to distill reasoning abilities from large language models (LLMs) into smaller student models by framing the task as a constrained reinforcement learning problem. It enforces a worst‑case constraint on the teacher’s log‑likelihood for every prefix of the reasoning chain, avoiding reward hacking and excessive teacher regularization. Experiments on mathematical reasoning and code generation show that this method improves the balance between accuracy and fidelity, achieving the highest rigorous reasoning success rate among evaluated settings.
arXiv:2610.00388v1 Announce Type: cross Abstract: Reinforcement learning enables large language model (LLM) agents to learn multi-step behaviors through interaction with their environments. However,...
arXiv:2610.00389v1 Announce Type: cross Abstract: Open-ended query generation lacks standard answers, thus necessitating an effective reward mechanism. Pointwise scoring rubrics provide limited infor...
arXiv:2610.00592v1 Announce Type: cross Abstract: In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next...
arXiv:2610.00758v1 Announce Type: cross Abstract: By learning transferable rewards, inverse reinforcement learning (IRL) enables counterfactual evaluation of agents under modified environments. Such...
arXiv:2610.00838v1 Announce Type: cross Abstract: Agentic reinforcement learning (RL) trains a large language model (LLM) to act over long, multi-step interactions. However, a single localized error...