Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2509.23140v2 Announce Type: replace Abstract: Recent advancements have endowed Large Language Models with impressive general reasoning capabilities. However, these reasoning models often perfor...
arXiv:2609.36851v1 Announce Type: new Abstract: End-to-end autonomous driving policies are commonly trained via imitation learning on logged demonstrations without observing the consequences of their...
arXiv:2607.28225v2 Announce Type: replace Abstract: Agentic vision-language models (VLMs), which interleave textual reasoning with explicit tool calls such as cropping and code-based image manipulati...
The paper introduces Traverse, an autonomous web‑search agent that manages its search process through three states—Rubric, Answer, and Verify—while using a Seal Memory tool for active context management. Reinforcement learning is employed to train the agent, but a training instability called Seal Collapse is mitigated by training only the final segment after context management. The resulting 35B model achieves state‑of‑the‑art performance on BrowseComp and related benchmarks, outperforming comparable open‑source systems.
The paper investigates how the quantity, source, and selection of prompts influence transfer in on‑policy distillation (OPD) between teacher and student models. It shows that a small set of well‑chosen prompts can achieve performance comparable to large prompt pools, but the effectiveness of prompts depends on the specific teacher‑student pair and target task. The study also finds that prompt utility is relational rather than intrinsic, and that targeted prompt selection does not consistently outperform random sampling.
The paper introduces a decision‑theoretic framework that splits a large language model’s decision loss into belief formation and action selection components. Using a synthetic benchmark, it evaluates how reinforcement‑learning interventions on beliefs, decisions, or both affect these components across three domains. The study finds that targeting a single component improves that part but may not transfer to others, while jointly targeting both improves both only when training and evaluation formats match.
arXiv:2607.28623v2 Announce Type: replace-cross Abstract: We present PAC-MAN, a perception-aware CBF-RL framework that couples control-barrier safety with deployment-realistic onboard sensing for who...
arXiv:2609.37175v1 Announce Type: new Abstract: Large language models (LLMs) have provided a unified interface for end-to-end combinatorial optimization (CO), but textual serialization alone may obsc...
arXiv:2609.37969v1 Announce Type: new Abstract: High-resolution video generation is expensive, as its cost grows rapidly with the number of spatiotemporal tokens. A practical alternative first genera...
arXiv:2605.18641v2 Announce Type: replace Abstract: Latent visual reasoning involves visual evidence more directly in multimodal reasoning by inserting continuous latent tokens before textual generat...
arXiv:2609.36254v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) are commonly trained with reinforcement learning (RL) to improve their generation of chain-of-thought (CoT) reasoning bef...
arXiv:2609.38016v1 Announce Type: new Abstract: Constrained Reinforcement Learning has recently gained increasing attention in the field of Safe Autonomous Driving, where the general mechanism is to...
arXiv:2609.36804v1 Announce Type: cross Abstract: Chinese Semantic Error Correction (CSEC) targets semantic errors in Chinese text, which are typically more subtle and complex than spelling and gramm...
arXiv:2609.37372v1 Announce Type: new Abstract: Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and can...
arXiv:2609.35880v1 Announce Type: new Abstract: Offline reinforcement learning (RL) has traditionally focused on learning policies for direct deployment under conservative objectives, where uncertain...
arXiv:2609.36116v1 Announce Type: new Abstract: We study how to build more capable general-purpose agents by extending large language models (LLMs) with native typed decision-making. We introduce Dya...
arXiv:2609.36250v1 Announce Type: new Abstract: Action chunking provides temporal abstraction in reinforcement learning by selecting short action sequences instead of individual actions, but many exi...
arXiv:2609.36393v1 Announce Type: cross Abstract: Traditional reinforcement learning (RL) techniques focus on maximizing expected cumulative reward, where each action assumes to take a constant unit...
arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
arXiv:2609.36642v1 Announce Type: new Abstract: Language-model agents are usually trained by reinforcement learning from one reward per episode, and privileged self-distillation enriches it by lettin...