Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

3,577 stories · RSS feed

arXiv AI

Jul 24

Evaluating Risks in Weak-to-Strong Alignment: A Bias-Variance Perspective

arXiv:2604. 25077v2 Announce Type: replace Abstract: Weak-to-strong alignment offers a promising route to scalable supervision, but it can fail when a strong model becomes confidently wrong on examples that lie in the weak model's blind spots.

By Hamid Osooli, Kareema Batool, Rick Gentry, Tiasa Singha Roy, Ashwin Gupta, Anirudha Ramesh

reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 24

Error Amplification Limits ANN-to-SNN Conversion in Continuous Control

arXiv:2601. 21778v3 Announce Type: replace-cross Abstract: Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs), avoiding further costly training.

By Zijie Xu, Zihan Huang, Yiting Dong, Kang Chen, Wenxuan Liu, Zhaofei Yu

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 24

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.

By Chenlong Yin, Runpeng Geng, Yanting Wang, Jinyuan Jia

llms agents reinforcement-learning efficiency benchmarks safety

arXiv AI

Jul 24

PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning

arXiv:2607. 21419v1 Announce Type: new Abstract: In long-horizon LLM agent reinforcement learning, weak policies often repeat similar failures, producing uninformative rollout trajectories and limiting effective policy optimization.

By Yipeng Shi, Zhipeng Ma, Yue Wang, Qitai Tan, Yang Li, Peng Chen, Zhengzhou Zhu

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 24

Wireless TokenCom: RL-Based Tokenizer Agreement for Multi-User Wireless Token Communications

arXiv:2602. 12338v2 Announce Type: replace Abstract: Token Communications (TokenCom) has recently emerged as an effective new paradigm, where tokens are the unified units of multimodal communications and computations, enabling efficient digital semantic- and goal-oriented communications in future wireless networks.

By Farshad Zeinali, Mahdi Boloursaz Mashhadi, Rahim Tafazolli

diffusion nlp reinforcement-learning multimodal

arXiv AI

Jul 24

Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs

arXiv:2509. 14257v3 Announce Type: replace-cross Abstract: Large Language Model agents achieve strong performance on multi-step reasoning and tool-use tasks, but their impressive capabilities typically rely on extremely large backbones.

By Yuanjie Lyu, Chengyu Wang, Jun Huang, Tong Xu

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jul 24

DynaMark: A Reinforcement Learning Framework for Dynamic Watermarking in Industrial Machine Tool Controllers

arXiv:2508. 21797v2 Announce Type: replace-cross Abstract: Industry 4.

By Navid Aftabi, Abhishek Hanchate, Satish Bukkapatnam, Dan Li

reinforcement-learning benchmarks

arXiv AI

Jul 24

ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents

arXiv:2606. 21262v2 Announce Type: replace Abstract: Reinforcement learning for multi-step LLM agents often relies on scalar rewards that indicate success but cannot explain why a trajectory is good or bad.

By Zihang Tian, Jingsen Zhang, Rui Li, Xiaohe Bo, Yuanzi Li, Xu Chen

llms agents reinforcement-learning safety

arXiv AI

Jul 24

Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections

arXiv:2607. 21488v1 Announce Type: cross Abstract: Coordinating autonomous vehicles at unsignalized intersections remains a critical challenge for multi-agent reinforcement learning (MARL) systems, which typically struggle with combinatorial action spaces, reliance on privileged information, or rigid agent designs.

By Gil Lifshits, Igal Bilik, Gilad Katz

rag agents reinforcement-learning benchmarks

arXiv AI

Jul 24

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

arXiv:2607. 21111v1 Announce Type: cross Abstract: Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected data must be removed after training.

By Chaofan Pan, Lingfei Ren, Xiangyu Jiang, Yanhua Li, Xuemei Cao, Xiangkun Wang, Hao Yu, Wei Wei, Xin Yang

agents reinforcement-learning robotics fine-tuning benchmarks safety

arXiv AI

Jul 24

Relative Value Learning

arXiv:2607. 21120v1 Announce Type: cross Abstract: In reinforcement learning, critics typically estimate absolute state values $V(s)$, estimating how good a particular situation is in isolation.

By Marc H\"oftmann, Jan Robine, Stefan Harmeling

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 24

Domyn-Small: A European 10B Reasoning Language Model

arXiv:2607. 20448v1 Announce Type: cross Abstract: We introduce Domyn-Small, a 10-billion-parameter open-weight reasoning language model released under the MIT license.

By Simone Angarano, Francesco Bertolotti, Federico D'Ambrosio, Michele Resta, Alessandro Rognoni, Nicol\`o Ruggeri, Dario Salvati, Andrea Valenti, Alberto Veneri, Martin Cimmino

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jul 24

Conflict Resolution under Degraded Surveillance in Air Corridors Using Multi-Agent Reinforcement Learning

arXiv:2607. 20547v1 Announce Type: new Abstract: Safe Advanced Air Mobility operations require aircraft to maintain separation when surveillance information is noisy, delayed, incomplete, or temporarily unavailable.

By Esrat Farhana Dulia, Syed Arbab Mohd Shihab, Caleb Adams, Ruben Del Rosario

agents reinforcement-learning safety

arXiv Machine Learning

Jul 24

The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works

arXiv:2607. 21273v1 Announce Type: new Abstract: Dense per-step supervision is an appealing remedy for sparse-reward, long-horizon LLM agents: reward the agent for predicting its next observation, and memory should follow.

By Yu Wang

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 24

From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

arXiv:2607. 21069v1 Announce Type: new Abstract: The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also serve as a training signal.

By Muntasir Adnan, Manile Srun, Carlos C. N. Kuhn

reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning

arXiv:2607. 20952v1 Announce Type: new Abstract: Latent, or silent, reasoning lets language models carry out intermediate computation in continuous vector space instead of words, and is widely assumed to function as an internal scratchpad the model actively consults during inference.

By Ishan S. Kshirsagar

llms reinforcement-learning fine-tuning

arXiv AI

Jul 24

CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

arXiv:2607. 20553v1 Announce Type: new Abstract: Memory Manager models are pivotal in agent systems.

By Yubo Wang, Qiuyu Zhao, Zenghui Sun, Shichao Dong, Jinsong Lan, Xiaoyong Zhu, Haoyang Li, Bo Zheng, Lei Chen

llms agents reinforcement-learning

arXiv AI

Jul 24

Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind

arXiv:2604. 11666v2 Announce Type: replace-cross Abstract: As large language models (LLMs) become the engine behind conversational systems, their ability to reason about the intentions and states of their dialogue partners (i.

By Hanqi Xiao, Vaidehi Patil, Zaid Khan, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal

llms agents reinforcement-learning safety

arXiv AI

Jul 24

TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework

arXiv:2511. 05385v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) utilizes external knowledge to augment Large Language Models' (LLMs) reliability.

By Chao Zhang, Yuhao Wang, Derong Xu, Haoxin Zhang, Yuanjie Lyu, Yuhao Chen, Shuochen Liu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, Enhong Chen

llms rag agents reinforcement-learning

arXiv AI

Jul 24

Training Large Language Models for Self-Explanation Faithfulness

arXiv:2607. 21090v1 Announce Type: cross Abstract: We propose a Reinforcement Learning (RL) method to directly optimize the faithfulness of self-explanations - the extent to which a model's generated reasoning accurately reflects its internal decision-making process.

By Yeoktatt Cheah, Mar\'ia P\'erez-Ortiz, Noah Y. Siegel, Oana-Maria Camburu

llms reinforcement-learning fine-tuning safety

Get the daily briefing