Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2504.19024v2 Announce Type: replace Abstract: We propose a novel k-step return estimation method (called KETCHUP) for Reinforcement Learning(RL)-based knowledge distillation (KD) in text genera...
arXiv:2601.16400v2 Announce Type: replace Abstract: Real-world visual question answering (VQA) may be context-dependent: an image-question pair could be under-specified, such that the correct answer...
The article discusses how glass-box deep reinforcement learning (DRL) can discover actuation strategies for high‑speed, nonlinear, unsteady flows without explicit models, potentially revealing mechanisms beyond classical control. It highlights the challenges of applying DRL to real high‑speed flows, including simulation cost, data efficiency, sensing in extreme environments, reward design, explainability, and generalization across Mach and Reynolds numbers. The authors argue that overcoming these constraints will require physics‑aware learning, robust real‑time operation, and extensive validation to translate DRL capabilities into reliable, certified technology.
Self-Credit is an agentic reinforcement learning method that transforms sparse outcome rewards into dense supervision signals through self-generated credit assignment and intra-group rescaling. The approach involves the policy model summarizing key evidence with citations, refining these citations by comparing actions across successful and failed rollouts, and reweighting advantages based on the rescaled credit. Experiments on BrowseComp-Plus, GAIA, and xBench-DeepSearch show significant performance gains over baseline models and other agentic RL methods.
arXiv:2602.00348v2 Announce Type: replace Abstract: Metal implants in MRI cause severe artifacts that degrade image quality and hinder clinical diagnosis. Traditional approaches address metal artifac...
The paper introduces LENS, a tool for black‑box performance analysis of large language model (LLM)–based network control systems. LENS extends deep reinforcement learning to identify performance gaps and challenging inputs for LLMs, revealing significant deficiencies in LLM‑driven traffic engineering and GPU allocation. It demonstrates that LENS can uncover gaps comparable to those found by white‑box analyzers for hand‑crafted heuristics.
The paper proposes an autonomous construction system that combines a large language model (ChatGPT‑5) with a reinforcement learning algorithm to control a mobile crane during retaining wall construction. An enhanced soft actor‑critic model plans crane actions, while robot kinematics ensure these actions are executable. The system also uses a custom embedding model to extract relevant construction data and generates progress updates, safety information, and reports.
The paper introduces SCAMPER, a system that trains a virtual rat model to exhibit realistic movements during both goal-directed tasks and random exploration. SCAMPER employs a variational architecture with two priors—a motion prior for control and a behavior prior for exploration—to generate natural behavior and effectively exploit task structure. The system can learn from sparse rewards, enabling realistic modeling of sparse-reward tasks used in experimental neuroscience.
TrajEvolve introduces a trajectory evolution paradigm with hindsight credit assignment to improve reinforcement learning for large language models on long-horizon tasks. It refines low-quality steps in historical trajectories to expose the policy to high-quality compositions that are rarely sampled, and reformulates step-level continuous credit assignment as a binary classification problem to identify redundant steps. Experiments on two datasets show that this approach outperforms existing methods.
arXiv:2607.11578v2 Announce Type: replace-cross Abstract: Deep learning for EEG-based seizure detection faces critical challenges: severe annotation scarcity and extreme class imbalance, where ictal...
WeGR is a unified generative retrieval framework that tackles two key challenges in industrial search: preserving collaborative signals in tokenization and mitigating reward sparsity in reinforcement learning. It introduces the Collaborative and Collapse-free Tokenizer (CoCo‑VAE) to embed query‑to‑item and item‑to‑item signals into continuous representations before quantization, while also refreshing underutilized codewords to maintain balanced codebook usage. For preference alignment, Dynamic Horizon Policy Optimization (DHPO) expands reward evaluation horizons to enable coarse‑to‑fine learning from SID prefixes to full items without auxiliary reward models. Extensive offline tests and a six‑week online A/B test on a large‑scale e‑commerce platform demonstrate that WeGR improves retrieval and ranking performance, yielding relative gains of 2.539% in orders, 1.508% in CTR, and 1.381% in CVR, and reduces CPU and GPU usage by 25.5% and 26.1% respectively.
arXiv:2609.34575v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) learns goal-directed policies from reward-free data, but in long-horizon tasks, goal-conditi...
The paper introduces SpaRLax, a sparse relaxation method for reinforcement learning that enhances soft Chain-of-Thought reasoning in large language models. By treating soft tokens as differentiable intermediate steps and applying an adaptive entmax sparsity technique, SpaRLax stabilizes training and improves performance across nine reasoning benchmarks. The method retains multiple plausible candidates when uncertain while truncating noise at confident steps.
arXiv:2610.27735v1 Announce Type: new Abstract: Recursive self-improvement, in which frontier AI models work on training the next generation, is quickly becoming a reality. However, the safety of thi...
arXiv:2601.09736v2 Announce Type: replace-cross Abstract: Linear Genetic Programming (LGP) is a powerful technique that allows for a variety of problems to be solved using a linear representation of...
arXiv:2610.27012v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model by sampling responses, scoring them with an automatic verifier, and upd...
The paper investigates whether compact AI agents can display strategic behavior in poker under hidden information and whether preference optimization can surpass static and zero‑shot policies. After evaluating a static baseline and three frozen language models (SmallThinker, Qwen3, Gemma‑3), the authors introduce Strategic Preference Optimization with KL‑anchored Equilibrium (SPOKE), which blends teacher‑relative value penalties, policy anchoring, risk constraints, and structured rationale supervision. SPOKE outperforms other methods on Kuhn and Leduc games, achieving the lowest mean exploitability and the highest six‑max top‑1 agreement while maintaining teacher‑consistent behavior.
arXiv:2610.27670v1 Announce Type: new Abstract: This paper introduces a novel framework for efficiently evaluating in-game sound designs using multimodal learning and reinforcement learning. Sound de...
arXiv:2610.27847v1 Announce Type: new Abstract: Scaling the training of Large Language Model (LLM) agents typically requires a large collection of executable environments, whose construction, mainten...
arXiv:2610.28121v1 Announce Type: new Abstract: Safe autonomous navigation of UASs requires tactical deconfliction to address conflicts that strategic coordination cannot prevent. This task is challe...