Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

Hugging Face Trending Papers

Jun 29

ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limiting both generation quality and generalization. To address this, we analyze the complementary roles of forward and reverse KL divergence (FKL/RKL) in distribution alignment from theoretical and empirical perspectives.

llms reinforcement-learning efficiency benchmarks safety

arXiv AI

Jun 29

The Minimal Search Space for Conditional Causal Bandits

arXiv:2502. 06577v3 Announce Type: replace-cross Abstract: Causal knowledge can be used to support decision-making problems.

By Francisco N. F. Q. Simoes, Itai Feigenbaum, Mehdi Dastani, Thijs van Ommen

reinforcement-learning

arXiv AI

Jun 29

Trust Region Masking for Long-Horizon LLM Reinforcement Learning

arXiv:2512. 23075v5 Announce Type: replace-cross Abstract: Policy gradient methods for Large Language Models optimize a policy $\pi_\theta$ via a surrogate objective computed from samples of a rollout policy $\pi_{\text{roll}}$.

By Yingru Li, Jiacai Liu, Jiawei Xu, Yuxuan Tong, Ziniu Li, Qian Liu, Baoxiang Wang

llms reinforcement-learning

arXiv AI

Jun 29

Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing

arXiv:2606. 27926v1 Announce Type: new Abstract: Geometry Problem Solving have increasingly adopt the neuro-symbolic paradigm, combining neural intuition with symbolic rigor.

By Can Li, Ting Zhang, Junbo Zhao, Hua Huang

agents reinforcement-learning multimodal

arXiv AI

Jun 29

Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking

arXiv:2604. 26360v2 Announce Type: replace-cross Abstract: Reinforcement learning from human feedback (RLHF) systems face a compounding alignment challenge: not only are learned reward models uncertain about unseen state-action pairs, but the human preference annotations they are trained on are themselves inconsistent, context-dependent, and noisy.

By Disha Singha

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 29

Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience

arXiv:2606. 27475v1 Announce Type: cross Abstract: Robots trained on real world data tend to be imprecise, slow, and brittle to perturbations.

By Raymond Yu, William Huey, Mustafa Mukadam, Anusha Nagabandi, Abhishek Gupta

reinforcement-learning robotics efficiency

arXiv Machine Learning

Jun 29

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting

arXiv:2510. 18874v3 Announce Type: replace Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.

By Howard Chen, Noam Razin, Karthik Narasimhan, Danqi Chen

llms reinforcement-learning fine-tuning

arXiv AI

Jun 29

Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning

arXiv:2606. 27483v1 Announce Type: new Abstract: Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remains fundamentally reactive in long-horizon tasks.

By Xuan Zhang, Zhijian Zhou, Lingfeng Qiao, Yulei Qin, Ke Li, Xing Sun, Xiaoyu Tan, Chao Qu, Yuan Qi

llms agents reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 29

Non-Linear Model-Based Sequential Decision-Making in Agriculture

arXiv:2509. 01924v4 Announce Type: replace-cross Abstract: Agricultural decision-making faces a dual challenge: sustaining high yields to meet global food security needs while reducing the environmental impacts of input use, including fertilizer losses and other agrochemical applications such as herbicides, insecticides, and fungicides.

By Sakshi Arya, Wentao Lin

reinforcement-learning

arXiv AI

Jun 29

Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF

arXiv:2606. 27580v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) in production does not always have a synchronous reward signal.

By Arnav Raj

reinforcement-learning safety

arXiv AI

Jun 29

ToE: A Hierarchical and Explainable Claim Verification Framework with Dynamic Multi-source Evidence Retrieval and Aggregation

arXiv:2606. 27736v1 Announce Type: new Abstract: The rapid spread of fake news poses increasing threats to information ecosystems, especially as AI-generated misinformation under Generative Engine Optimization (GEO) poisoning allows adversarially crafted content to be systematically surfaced by retrieval systems, contaminating LLM reasoning.

By Zhaoqi Wang, Zijian Zhang, Kun Zheng, Zhen Li, Xin Li, Chunlei Li, Jiamou Liu

llms agents reinforcement-learning

arXiv Machine Learning

Jun 29

Learning Peer Influence Probabilities with Linear Contextual Bandits

arXiv:2510. 19119v2 Announce Type: replace Abstract: In networked environments, it is common for users to share recommendations about content, products, services, and possible courses of action.

By Ahmed Sayeed Faruk, Mohammad Shahverdikondori, Elena Zheleva

diffusion reinforcement-learning

arXiv Machine Learning

Jun 29

PerturbCellRL: Verifier-Guided Reinforcement Learning for Single-Cell Perturbation Prediction

arXiv:2606. 27752v1 Announce Type: new Abstract: Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond transcriptionally to interventions.

By Dongxia Wu, Mingyu Li, Yuhui Zhang, Anurendra Kumar, Emma Lundberg, Serena Yeung-Levy, Emily B. Fox

reinforcement-learning benchmarks safety

arXiv AI

Jun 29

PEBS: Per-rater Empirical-Bayes Shrinkage for RLHF Reward-Model Calibration

arXiv:2606. 27578v1 Announce Type: cross Abstract: Reward models for Reinforcement Learning from Human Feedback (RLHF) pool preferences across thousands of annotators and fit one global affine calibrator, collapsing raters with systematically different rating-scale offsets and slopes into a single average-rater fit that does not match any individual annotator.

By Arnav Raj

reinforcement-learning

arXiv AI

Jun 29

MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy

arXiv:2606. 27652v1 Announce Type: new Abstract: We find that explicit reasoning does not necessarily translate into better multimodal emotion recognition (MER) accuracy, even though it makes predictions more interpretable.

By Zhiyuan Han, Beier Zhu, Wenwen Tong, Chengwei Qin, Xinyi Wang, Jiayu Zhang, Jiangnan Chen, Hewei Guo, Dongchuan Ran, Lewei Lu, Xun Yang

reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 29

TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

arXiv:2506. 10355v2 Announce Type: replace Abstract: Many real-world applications collect data in a streaming environment, where learning tasks are encountered sequentially.

By Yu-Yang Qian, Yuan-Ze Xu, Zhen-Yu Zhang, Peng Zhao, Zhi-Hua Zhou

llms nlp reinforcement-learning

arXiv AI

Jun 29

Tandem Reinforcement Learning with Verifiable Rewards

arXiv:2606. 28166v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capability of large language models, reaching expert or even superhuman performance in domains such as competition math.

By Difan Jiao, Raghav Singhal, Robert West, Ashton Anderson

llms agents reinforcement-learning

arXiv AI

Jun 29

Agentic Episodic Control

arXiv:2506. 01442v2 Announce Type: replace Abstract: Reinforcement learning (RL) remains fundamentally limited by poor data efficiency and weak generalization.

By Xidong Yang, Wenhao Li, Junjie Sheng, Yun Hua, Haosheng Chen, Chuyun Shen, Xiangfeng Wang

llms agents reinforcement-learning

arXiv Machine Learning

Jun 29

Effects of relational graph modularity and depth on the learning performance of neural networks

arXiv:2507. 10005v2 Announce Type: replace Abstract: In recent years, graph-based machine learning techniques, such as reinforcement learning and graph neural networks, have garnered significant attention.

By Yash Arya, Sang Hoon Lee

computer-vision reinforcement-learning

arXiv AI

Jun 29

Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents

arXiv:2606. 27472v1 Announce Type: cross Abstract: Large language model (LLM) agents operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised.

By Vedant Patel

llms agents reinforcement-learning fine-tuning

Get the daily briefing