Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2609.26146v1 Announce Type: new Abstract: Risk monitoring in multi-agent systems is commonly built on a per-state primitive that scores each state independently and selects the top K. Under cro...
arXiv:2504.20106v4 Announce Type: replace Abstract: Ensuring that large language models (LLMs) are both helpful and harmless is a critical challenge, as overly strict constraints can lead to excessiv...
arXiv:2608.01069v2 Announce Type: replace Abstract: Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable in...
arXiv:2510.17881v4 Announce Type: replace Abstract: Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We int...
The paper investigates how reinforcement learning can unintentionally obscure the chain‑of‑thought (CoT) reasoning in vision‑language models, making their internal reasoning less traceable. By analyzing activation patterns, the authors show that template‑associated activations become less distinguishable during RL and that targeted interventions can mitigate this effect. They introduce TAME, a method that uses sparse autoencoders to suppress these problematic activations while still encouraging accurate behavior, achieving significant gains in CoT monitorability across multiple datasets and model families.
WeightBridge is a lightweight library that streamlines weight transfer between trainers and rollout generators in reinforcement learning systems, particularly for large language models. It automatically maps trainer and rollout weight layouts, then performs redundancy‑free, load‑balanced transfers while supporting various synchronization modes. Experiments show that WeightBridge can cut GPU stall time by up to 42× compared to leading open‑source RL frameworks, and it was easily integrated into two different frameworks by a coding agent.
KwaiMind is a commercial image editing system that combines general editing capabilities with e-commerce specialization. It uses an agent-based data engine with 1.8 million editing pairs and a multimodal diffusion transformer trained through pre‑training, fine‑tuning, preference optimization, and online reinforcement learning. The system is guided by a vision‑language judge and specialized rewards for click‑through rate, text rendering, and product consistency, and it achieves top scores on ImgEdit, GEdit, REDEdit, and the new Ecom‑Bench, while improving predicted and actual CTR in offline and online experiments.
arXiv:2605.11151v3 Announce Type: replace Abstract: Offline-to-online reinforcement learning (RL) improves sample efficiency by leveraging pre-collected datasets prior to online interaction. A key ch...
Video-HopChain introduces a new dataset of 22,550 multi‑hop video questions over 13,378 videos, each question consisting of three to six yes/no sub‑questions whose integer answers sum to a verifiable reward. Training a Qwen3‑VL‑8B model with GRPO on this dataset improves performance across eight video‑understanding benchmarks from 55.4 to 57.9, and the addition of Confidence‑Gated Exploration (CGE) raises the mean to 59.3. The authors release the dataset, checkpoint, and training code for further research.
arXiv:2609.24526v2 Announce Type: replace Abstract: Physical AI requires models to ground visual and linguistic understanding in real-world environments while accounting for environmental constraints...
Language-model agents increasingly face long-horizon tasks with evolving state, interdependent decisions, and delayed outcomes. Scaling their training requires diverse agentic environments, dependable...
Language-model agents now run the whole of quantitative factor research: they propose investment factors, backtest them, select the survivors and retire them. We ask which of those jobs an agent shoul...
Hindsight relabeling which retroactively replacing a transition's goal with the outcome the agent actually achieved is an effective tool for improving sample-efficiency in Reinforcement Learning (RL)....
Offline reinforcement learning and off-policy evaluation evaluates dynamic treatment rules based on retrospectively collected data prior to deployment. In recent AI applications, state and reward info...
Using large language models (LLMs) to simulate diverse human populations has the potential to transform many aspects of computational social science, yet many evaluations score the average response ra...
The paper introduces SCOPE, a method that post‑trains computer‑use agents to balance task completion with safety by conditioning actions on environmental risk. It combines supervised fine‑tuning on three trajectory types—capability demonstrations, safe continuations, and explicit refusals—followed by reinforcement learning to improve performance. Experiments starting from Qwen3.5‑9B show that SCOPE‑RL achieves high task success and attack‑avoidance rates, outperforming other agents on OSWorld and OS‑BLIND benchmarks.
The paper introduces a two-part neural framework for generating and analyzing forced three-dimensional incompressible Navier–Stokes flow. Part I focuses on a physics‑informed neural model that produces structured external‑force trajectories, optimizes them via differentiable PDE rollouts or PPO‑Clip, and validates selected forcings through fixed‑force replay. Part II provides a mathematical certification layer that separates neural candidate discovery from continuum analysis, derives integrated reciprocal‑vorticity criteria implying Riccati‑type growth and finite‑time loss of smooth continuation, and establishes a conditional positive‑probability closure for a nondegenerate neural output law, completing the proof at the continuum level.
The paper introduces RIBA, a reinforcement‑learning inspired black‑box adversarial attack that generates perturbations for neural networks with fewer queries than existing methods. RIBA achieves a 25.4% reduction in median queries on ResNet‑18/Cifar10 and a 22.5% reduction on Vit‑B/16/ImageNet, while matching white‑box attack performance on an adversarially trained model.
The paper argues that the observed crossover in pass@$k$ performance between reinforcement learning with verifiable rewards (RLVR) and its base model is not always statistically confirmed. By constructing confidence bands across sampling budgets and performing power analyses, the authors show that many reported crossovers lack statistical support and that more prompts can improve detection more than more answers per prompt. They further demonstrate that RLVR’s effect on a prompt is better described as a conditional distribution (a Markov kernel) rather than a single curve, allowing predictions of crossovers in new data and clarifying how losses on difficult prompts can overturn early gains.
The paper introduces Chart‑RVR, a reinforcement‑learning framework that trains chart‑reasoning agents to produce monitorable, verifiable outputs. It decomposes reasoning into three auditable stages—Structure, Evidence, and Derivation—allowing stakeholders to trace how the model reads the chart, extracts data, and computes the answer. Experiments on six benchmarks show that Chart‑RVR matches or exceeds state‑of‑the‑art accuracy while delivering rationales that are far more verifiable and evidence‑grounded than existing methods.