Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.01133v1 Announce Type: cross Abstract: Scaling reasoning typically spends more compute on reinforcement learning (RL) or on inference. We show that a completed RL training history can yiel...
arXiv:2610.01789v1 Announce Type: cross Abstract: Reinforcement learning based post training of diffusion models, such as Denoising Diffusion Policy Optimization (DDPO), optimizes a reverse diffusion...
arXiv:2610.02015v1 Announce Type: cross Abstract: Recent advances in LLM reasoning models---driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)-...
arXiv:2610.02039v1 Announce Type: cross Abstract: Recent years have witnessed the rapid adoption of reinforcement learning (RL) in large language model (LLM) post-training, with substantial gains in...
arXiv:2610.02198v1 Announce Type: cross Abstract: Several state-of-the-art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic....
arXiv:2610.01260v1 Announce Type: cross Abstract: Quadrupedal locomotion requires balancing conflicting objectives such as command tracking, stability, and energy efficiency, yet conventional reinfor...
arXiv:2610.01413v1 Announce Type: cross Abstract: Reinforcement learning (RL) algorithms frequently compare probability distributions, such as state visitation distributions induced by policies and e...
The paper investigates how reinforcement learning post‑training of large language models (LLMs) tends to sharpen existing behaviors, improving single‑shot accuracy but reducing solution coverage. It shows that pre‑trained LLMs, when paired with a lightweight inference harness, can outperform post‑trained models in coverage for agentic tasks that require multi‑turn tool use. The authors introduce the Sharpening Tax metric to quantify this trade‑off, analyze its prevalence across 14 model pairs and 42 benchmark cases, and propose a Bayesian sampler, PTGS, that mitigates the tax by adapting sampling temperature to prompt difficulty.
arXiv:2610.01546v1 Announce Type: cross Abstract: Primal-dual hybrid gradient (PDHG) methods solve large-scale linear programs (LPs) using GPU-friendly matrix-vector products and projections, but the...
arXiv:2510.24046v2 Announce Type: replace-cross Abstract: Existing tabular data generation methods primarily focus on matching statistical distributions between real and synthetic data, often overloo...
arXiv:2604.14908v2 Announce Type: replace-cross Abstract: We study downlink beam and rate adaptation in a multi-user mmWave MISO system where multiple base stations (BSs), each using analog beamformi...
arXiv:2604.24957v3 Announce Type: replace-cross Abstract: Scaling test-time compute has emerged as a powerful mechanism for enhancing Large Language Model (LLM) performance. However, standard post-tr...
The paper introduces POISE, a reinforcement learning algorithm that uses a model’s internal states as a value estimator to reduce variance in reinforcement learning with verifiable rewards (RLVR). By employing a lightweight probe that reads internal signals during the forward pass, POISE predicts baselines online and uses a cross‑rollout construction to keep gradients unbiased. Experiments on Qwen3‑4B and OLMo3‑7B‑Instruct‑DPO across six domains show POISE outperforms existing RLVR baselines, offering more stable training and a value model that generalizes across tasks and scales with the policy.
Selection-Based Structured Reasoning (SSR) is a framework that replaces free-form reasoning with the selection of pre-specified natural‑language reasoning candidates, allowing multimodal agents to choose from a set of reusable high‑level reasoning traces. By scoring these candidates in parallel using a shared context KV cache, SSR eliminates the need for an auxiliary task head and reduces inference cost. Experiments on seven multimodal search benchmarks with 2B and 4B models show that SSR maintains competitive success rates while cutting per‑turn reasoning latency by over 90% and overall inference latency by 28–54%.
DashVMC is a system that learns a compact, action‑conditioned world model from about two hours of recorded Geometry Dash gameplay. The model is used to train a controller via behavioural cloning and Proximal Policy Optimization entirely in frozen‑model rollouts, achieving longer survival than the initial policy on all tested levels. At deployment, the system runs a 60‑Hz capture‑to‑action loop on a consumer GPU, demonstrating real‑time control despite imperfect long‑horizon fidelity.
arXiv:2610.00661v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by denoising a sequence or successive blocks, allowing several tokens to be revealed in paralle...
arXiv:2610.01352v1 Announce Type: new Abstract: Open multimodal reasoning models have benefited from large-scale reasoning supervision, yet reliable post-training remains challenging due to uneven da...
Open multimodal reasoning models have benefited from large-scale reasoning supervision, yet reliable post-training remains challenging due to uneven data quality, inefficient supervision construction,...
The paper introduces Rita, a reinforcement learning framework that addresses thinking drift in vision‑language models by enforcing consistency between reasoning and answers. Rita employs two reasoning‑label‑free rewards—thinking and consistency rewards—derived from the conditional probability of reference answers, and uses a difficulty‑aware data filtering strategy to select informative samples for training. Experiments on EgoIntention and RefEgo‑Int benchmarks demonstrate that Rita outperforms both supervised fine‑tuning and vanilla RL‑fine‑tuned approaches.
The paper introduces Diffusion-Augmented Markov Decision Processes (DA‑MDPs), a framework that extends Maximum Entropy Reinforcement Learning to diffusion-based policies. DA‑MDPs treat each reverse‑diffusion step as an RL decision, deriving a tractable reverse‑KL bound that decomposes across denoising transitions and yields diffusion‑augmented soft rewards, value functions, and policy objectives. The authors implement this framework with PPO, REPPO, and a maximum‑entropy WPO variant, showing improved continuous‑control performance, higher success rates on manipulation tasks, and memory‑efficient training with action chunking.