Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,685 stories · RSS feed

arXiv AI

Oct 15

Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison

arXiv:2610.18415v1 Announce Type: cross Abstract: Large Language Models (LLMs) are aligned to human preferences, after pre-training and supervised fine-tuning steps. However, standard RL alignment st...

By Pratuat Amatya, Vinay Setty, V Venktesh, Avishek Anand

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Oct 15

ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models

arXiv:2610.18596v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training improves robot foundation models (RFMs) beyond their demonstrations, but real-world deployment demands more...

By Changyeon Kim, Kimin Lee, Yashraj Narang, Yijie Guo

reinforcement-learning robotics

arXiv AI

Oct 15

DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization

arXiv:2610.18609v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has become a widely adopted approach for aligning vision-language models (VLMs), yet the security of multimodal...

By Chengtao Jian, Kai Yang

llms reinforcement-learning multimodal safety

arXiv AI

Oct 15

Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria

arXiv:2511.17937v2 Announce Type: replace Abstract: Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are i...

By Kartik Garg, Shourya Mishra, Kartikeya Sinha, Ojaswi Pratap Singh, Ayush Chopra, Kanishk Rai, Ammar Sheikh, Raghav Maheshwari, Aman Chadha, Vinija Jain, Amitava Das

llms reinforcement-learning safety

arXiv AI

Oct 15

Toward Agents That Reason About Their Computation

arXiv:2510.22833v3 Announce Type: replace Abstract: While reinforcement learning agents can achieve superhuman performance in many complex tasks, they typically do not become more computationally eff...

By Adrian Orenstein, Jessica Jiajing Chen, Gwyneth Anne Delos Santos, Bayley Sapara, Michael Bowling

agents reinforcement-learning

arXiv AI

Oct 15

e1: Learning Adaptive Control of Reasoning Effort

arXiv:2510.27042v3 Announce Type: replace Abstract: Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users m...

By Michael Kleinman, Matthew Trager, Alessandro Achille, Wei Xia, Stefano Soatto

reinforcement-learning

arXiv AI

Oct 15

Mitigating Distribution Sharpening in Math RLVR via Distribution-Aligned Hint Synthesis and Backward Hint Annealing

arXiv:2604.07747v2 Announce Type: replace Abstract: In reinforcement learning with verifiable rewards (RLVR), challenging math questions whose sampled groups remain uniformly incorrect yield no infor...

By Pei-Xi Xie, Che-Yu Lin, Cheng-Lin Yang

llms reinforcement-learning benchmarks

arXiv AI

Oct 15

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

arXiv:2605.08374v4 Announce Type: replace Abstract: Episodic memory allows LLM agents to accumulate and retrieve experience, but current methods treat each memory independently, i.e., evaluating retr...

By Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Bo Tang, Weinan Zhang, Muning Wen

llms agents reinforcement-learning benchmarks

arXiv AI

Oct 15

VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization

arXiv:2503.04940v3 Announce Type: replace-cross Abstract: Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more close...

By Mohammad Mahdi Samiei Paqaleh, Mehdi Jamalkhah, Mahdieh Soleymani Baghshah

agents reinforcement-learning efficiency safety

arXiv AI

Oct 15

Flow Policy Gradients for Robot Control

arXiv:2602.02481v2 Announce Type: replace-cross Abstract: Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on di...

By Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa

diffusion reinforcement-learning robotics fine-tuning

arXiv AI

Oct 15

DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay

arXiv:2603.16157v2 Announce Type: replace-cross Abstract: While Reinforcement Learning (RL) enhances Large Language Model reasoning, on-policy algorithms like GRPO are sample-inefficient as they disc...

By Long Li, Zhijian Zhou, Tianyi Wang, Weidi Xu, Zuming Huang, Wei Chu, Zhe Wang, Shirui Pan, Chao Qu, Yuan Qi

llms reinforcement-learning benchmarks data-engineering

arXiv AI

Oct 15

AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning

arXiv:2605.06149v2 Announce Type: replace-cross Abstract: The discount factor in reinforcement learning controls both the effective planning horizon and the strength of bootstrapping, yet most deep R...

By Yaomin Wang, Jianting Pan, Ran Tian, Xiaoyang Li, Yu Zhang, Hengle Qin, Tianshu YU

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Oct 15

PairIT: Autoregressive Transformers for Low-Data Molecule Optimization

arXiv:2610.19123v1 Announce Type: new Abstract: Generating novel molecules with desired properties is a central challenge in drug discovery. Although foundational work focused on de novo generation,...

By Hannah Lawrence, Bodhi P. Vani, Ji Won Park, Natasa Tagasovska, Samuel Don Stanton, Andrew Martin Watkins, Michael Maser, Ewa M. Nowara

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Oct 15

What Information Buys in Active Inference

arXiv:2610.19046v1 Announce Type: new Abstract: Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscie...

By Melih Kandemir, Mustafa Mert \c{C}elikok

agents reinforcement-learning benchmarks

arXiv Machine Learning

Oct 15

Contextual Credit Policy Optimization

arXiv:2610.19103v1 Announce Type: new Abstract: Group-based reinforcement learning enables large language model agents to learn from sparse outcome rewards in long-horizon tasks. Recent methods deriv...

By Huiwen Han, Ziyao Zeng, Zihao Zhu, Sisong Bei, Hao Wang, Hongyu Xu, Yan Han, Degui Zhi, Zhiwen Fan

llms agents reinforcement-learning

arXiv Machine Learning

Oct 15

Verifier-Grounded Agentic RL for Provably Correct Code

arXiv:2610.19298v1 Announce Type: new Abstract: Reinforcement learning for code usually rewards passing tests, which a model can do while remaining wrong on the inputs the tests do not cover. A progr...

By George Ma, Yuhao Zhang, Bing He, Zeren Shui, Ferhat Erata, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou

llms agents reinforcement-learning

arXiv Machine Learning

Oct 15

MIPS: Mitigating Mode Collapse in Reinforcement Learning for Large Outcome Spaces

arXiv:2610.19457v1 Announce Type: new Abstract: Expected-reward maximization in reinforcement learning concentrates probability mass on a few outcomes, even in tasks that require many distinct, high-...

By Armaan Sandhu, Abhijeet Sinha, Dianbo Liu

reinforcement-learning

arXiv Machine Learning

Oct 15

Comprehensive Comparisons of Reinforcement Learning Algorithms in Autonomous Driving

arXiv:2610.19429v1 Announce Type: new Abstract: This paper presents a comprehensive comparison of four representative reinforcement learning (RL) methods in autonomous vehicle (AV) decision-making, i...

By Dechen Xie, Mushuang Liu

agents reinforcement-learning robotics safety

arXiv Machine Learning

Oct 15

Function-Space Temporal Self-Distillation for Policy Optimization

arXiv:2610.19687v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over respon...

By Aryan Das, Arnab Bhattacharya, Vinay Kumar Verma

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 15

Bayesian Learning to Select Fair Classifiers

arXiv:2610.20514v1 Announce Type: new Abstract: Evaluations of classifiers (AI or human or agents) often assume a batch setting in which every agent produces labels on every example in a test set, wh...

By Saatvik Kher, Hyungrok Do, Judy Zhong, Padhraic Smyth

llms agents reinforcement-learning safety

Get the daily briefing