Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2609.05820v1 Announce Type: new Abstract: We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online setting with limited feedback. We...
arXiv:2609.06107v1 Announce Type: new Abstract: Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which do...
arXiv:2609.07303v1 Announce Type: new Abstract: Current paradigms for training language models via reinforcement learning rely heavily on sparse outcome rewards. However, as we pursue tasks that requ...
arXiv:2609.05808v1 Announce Type: cross Abstract: In situ adjoint training extracts parameter gradients directly from measurement, but has so far been limited to reciprocal or restricted systems. Her...
arXiv:2609.09300v1 Announce Type: new Abstract: Video understanding demands a convergence of complementary capabilities across perception, temporal understanding, and complex reasoning, which are dif...
arXiv:2609.09578v1 Announce Type: cross Abstract: Urban planning is a real-world spatial optimization problem that requires selecting feasible actions from large candidate spaces under practical obje...
AgenticGen is a reward‑guided framework for generating advertising videos that splits the task into strategy selection and draft generation, allowing online business feedback to supervise each stage. It learns performance‑based rewards from accumulated online metrics and rubric‑based rewards aligned with human quality standards, then applies DPO and GRPO to refine policies. Offline tests confirm the reward models, and online A/B tests on TikTok show significant gains in CTR, CVR, and advertising value over a baseline.
arXiv:2609.09905v1 Announce Type: cross Abstract: Preference alignment for flow and diffusion models now spans online reinforcement learning and offline preference optimization, but the relation betw...
arXiv:2604.10701v2 Announce Type: replace-cross Abstract: Credit assignment is a central challenge in reinforcement learning (RL). Classical actor-critic methods address this challenge through fine-g...
The paper investigates the use of variational quantum circuits (VQCs) in hierarchical reinforcement learning (HRL). It shows that a hybrid HRL agent incorporating a quantum feature extractor can outperform classical baselines with fewer parameters, but using VQCs for option-value estimation hampers learning. The study also explores how different quantum circuit designs influence performance and proposes design principles for efficient hybrid HRL agents.
arXiv:2609.09158v1 Announce Type: cross Abstract: We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D pat...
arXiv:2609.09075v1 Announce Type: cross Abstract: In reinforcement learning with verifiable rewards (RLVR) trained with group relative policy optimization (GRPO), the KL-free reward-advantage term st...
arXiv:2609.08211v1 Announce Type: new Abstract: Real-world Multi-Objective Reinforcement Learning (MORL) often suffers from sparse rewards, reward conflicts, and late-stage reward tug-of-war, causing...
arXiv:2609.05576v1 Announce Type: new Abstract: The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across statefu...
arXiv:2609.06882v1 Announce Type: cross Abstract: Diffusion policies offer a powerful and expressive parameterization for continuous control. Yet, their integration with reinforcement learning remain...
The paper introduces Multi-Negative Direct Preference Optimisation (MDPO), a pairwise objective that compares the correct entity with all valid rejected candidates for each mention, extending the single-negative approach used in prior work. MDPO retains the Bradley‑Terry formulation of Direct Preference Optimisation while leveraging the full candidate set through masked, length‑normalised sequence scores. Experiments on French, German, English, Swedish, and Finnish historical newspaper datasets (hipe‑2020 and newseye) show that MDPO outperforms both supervised fine‑tuning and single‑negative DPO, especially for NIL mentions, semantic ambiguity, OCR noise, and historically challenging names, and highlight candidate retrieval as a key bottleneck.
The paper proposes a CSI‑free hierarchical multi‑agent reinforcement learning framework for controlling reconfigurable reflective surfaces in millimeter‑wave networks. By replacing per‑element channel estimation with user localization data, the system uses a two‑tier neural architecture: a high‑level controller for discrete user‑to‑reflector assignments and low‑level controllers that optimize continuous focal points via MAPPO under a CTDE scheme. Deterministic ray‑tracing tests show RSSI gains of up to 7.79 dB over centralized PPO baselines and robust performance with sub‑meter localization errors for multiple users and reflector arrays.
arXiv:2609.07247v1 Announce Type: new Abstract: Scaling the interaction horizon-the maximum number of environment interactions per episode-improves LLM agents on long-horizon tasks, and curriculum-ba...
The paper introduces KTO, a new loss function that aligns large language models with human preferences by directly maximizing a prospect-theoretic utility function rather than log-likelihood of preferences. It builds on the idea that existing alignment objectives implicitly use human-aware losses (HALOs) that reflect biases such as loss aversion. KTO achieves performance comparable to or better than preference-based methods across 1B to 30B parameter models while only requiring a binary desirability signal, and it highlights that no single HALO is universally best, emphasizing the importance of choosing the right inductive biases for a given setting.
The paper introduces T2T (Thickening-to-Thinning), a dynamic reward framework for large language models that mimics human learning by separating exploration and consolidation phases. During incorrect attempts, T2T encourages exploration to broaden the search space, while after correct solutions it applies length penalties to promote concise reasoning. Experiments on mathematical benchmarks across five mainstream LLMs show that T2T outperforms standard GRPO and recent baselines, improving overall reasoning performance.