← Back to all news

Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,679 stories · RSS feed

arXiv AI
3d ago

Large language model and reinforcement learning based autonomous construction and progress updates

The paper proposes an autonomous construction system that combines a large language model (ChatGPT‑5) with a reinforcement learning algorithm to control a mobile crane during retaining wall construction. An enhanced soft actor‑critic model plans crane actions, while robot kinematics ensure these actions are executable. The system also uses a custom embedding model to extract relevant construction data and generates progress updates, safety information, and reports.

By Yifei Xiao, Huasheng Lin, T. Y. Yang
llmsragagentsreinforcement-learningroboticssafety
More like this →
arXiv AI
3d ago

A latent control model for realistic rodent motion

The paper introduces SCAMPER, a system that trains a virtual rat model to exhibit realistic movements during both goal-directed tasks and random exploration. SCAMPER employs a variational architecture with two priors—a motion prior for control and a behavior prior for exploration—to generate natural behavior and effectively exploit task structure. The system can learn from sparse rewards, enabling realistic modeling of sparse-reward tasks used in experimental neuroscience.

By Aidan Sirbu, Charles Y. Zhang, Yuanjia Yang, Bence P. \"Olveczky, Talmo D. Pereira, Blake A. Richards
agentsreinforcement-learningrobotics
More like this →
arXiv AI
3d ago

TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment

TrajEvolve introduces a trajectory evolution paradigm with hindsight credit assignment to improve reinforcement learning for large language models on long-horizon tasks. It refines low-quality steps in historical trajectories to expose the policy to high-quality compositions that are rarely sampled, and reformulates step-level continuous credit assignment as a binary classification problem to identify redundant steps. Experiments on two datasets show that this approach outperforms existing methods.

By Yi Wen, Hao Chen, Wanyu Wang, Maolin Wang, Pengyue Jia, Derong Xu, Huize Tan, Yingyi Zhang, Wenlin Zhang, Weihong Luo, Xiku Du, Xiangyu Zhao
llmsreinforcement-learning
More like this →
arXiv AI
3d ago

WeGR: Towards Unified Generative Retrieval via Behavior-aware Tokenization and Preference Alignment

WeGR is a unified generative retrieval framework that tackles two key challenges in industrial search: preserving collaborative signals in tokenization and mitigating reward sparsity in reinforcement learning. It introduces the Collaborative and Collapse-free Tokenizer (CoCo‑VAE) to embed query‑to‑item and item‑to‑item signals into continuous representations before quantization, while also refreshing underutilized codewords to maintain balanced codebook usage. For preference alignment, Dynamic Horizon Policy Optimization (DHPO) expands reward evaluation horizons to enable coarse‑to‑fine learning from SID prefixes to full items without auxiliary reward models. Extensive offline tests and a six‑week online A/B test on a large‑scale e‑commerce platform demonstrate that WeGR improves retrieval and ranking performance, yielding relative gains of 2.539% in orders, 1.508% in CTR, and 1.381% in CVR, and reduces CPU and GPU usage by 25.5% and 26.1% respectively.

By Jiajun Han, Guoxin Ma, Felix, Liangsheng Xu, Ruiqi Yu, Yingdi Guo, Zongbing Wang, Xinhua Ye, Chenyan Hua, Xin Xie, Pengcheng Yang, Haijun Wu
nlpreinforcement-learningefficiencysafety
More like this →
arXiv AI
3d ago

Sparse RL Relaxation for LLM Reasoning

The paper introduces SpaRLax, a sparse relaxation method for reinforcement learning that enhances soft Chain-of-Thought reasoning in large language models. By treating soft tokens as differentiable intermediate steps and applying an adaptive entmax sparsity technique, SpaRLax stabilizes training and improves performance across nine reasoning benchmarks. The method retains multiple plausible candidates when uncertain while truncating noise at confident steps.

By Jiabin Fan, Matthias Lindemann, Zita Marinho, Andr\'e FT Martins, Lili Mou
llmsragreinforcement-learningefficiencybenchmarks
More like this →
arXiv AI
3d ago

Teaching AI Agent to Play Poker Strategically Through Preference Optimization

The paper investigates whether compact AI agents can display strategic behavior in poker under hidden information and whether preference optimization can surpass static and zero‑shot policies. After evaluating a static baseline and three frozen language models (SmallThinker, Qwen3, Gemma‑3), the authors introduce Strategic Preference Optimization with KL‑anchored Equilibrium (SPOKE), which blends teacher‑relative value penalties, policy anchoring, risk constraints, and structured rationale supervision. SPOKE outperforms other methods on Kuhn and Leduc games, achieving the lowest mean exploitability and the highest six‑max top‑1 agreement while maintaining teacher‑consistent behavior.

By Phurinat Polasa, Thanapon Thanadunpremdet, Phatthanasak Kraiduang, Saksorn Ruangtanusak
llmsagentsreinforcement-learningfine-tuning
More like this →