Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,685 stories · RSS feed

arXiv Machine Learning

Oct 15

From Causal Graphs to Markovian States: Multi-Order State Exposure for Deep RL

arXiv:2605.07057v2 Announce Type: replace Abstract: Online reinforcement learning (RL) relies on the Markov property for guaranteed performance, but real-world applications often lack well-defined st...

By Jiamin Xu, Jacqueline Maasch, Kyra Gan

reinforcement-learning benchmarks

arXiv Computation and Language

Oct 15

MemDuet: Learning Dual-Process Memory for Long-Context Reasoning

arXiv:2610.19928v1 Announce Type: new Abstract: Reasoning over infinitely long documents is becoming a reality with recurrent memory agents. However, increasing context length exacerbates the trade-o...

By Jiangcheng Song, Haodong Jing, Yulin Mu, Lesong Tao, Jixin Wang, Jin Cui, Tianjing Ren, Runxuan Xie, Yongqiang Ma, Nanning Zheng

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Computation and Language

Oct 15

Dual-Evolving Reinforcement Learning for Automated Reward Design

arXiv:2512.13399v3 Announce Type: replace-cross Abstract: Crafting effective reward signals remains a central challenge in reinforcement learning (RL) for complex reasoning. Existing methods for auto...

By Sitao Cheng, Tianle Li, Xuhan Huang, Xunjian Yin, Difan Zou

agents reinforcement-learning robotics benchmarks

arXiv AI

Oct 15

Counterfactual Rollouts for Offline Contrastive Goal-Conditioned Reinforcement Learning

arXiv:2610.14953v1 Announce Type: cross Abstract: Contrastive reinforcement learning (CRL) learns goal-conditioned values by associating state--action pairs with the states subsequently reached, assu...

By Tingrui Huang, Ai Bo, Junzhe Zhang

reinforcement-learning

arXiv AI

Oct 15

RewardHarness: Self-Evolving Agentic Post-Training

arXiv:2605.08703v2 Announce Type: replace Abstract: Aligning generative models with human preferences remains a central challenge. While RLHF is widely adopted, it relies on large-scale preference da...

By Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, Changqian Yu, Ping Nie, Wenhu Chen, Kelsey R. Allen

llms agents reinforcement-learning

arXiv Machine Learning

Oct 15

AgentWorld: A Data Factory for Training LLM Agents from Multi-Turn Interactions

arXiv:2606.09138v2 Announce Type: replace Abstract: Training LLMs to act as capable agents requires learning from multi-turn interactions with environments. However, multi-turn interaction trajectori...

By Daoyu Wang, Mingyue Cheng, Qingchuan Li, Shuo Yu, Jie Ouyang, Qi Liu, Enhong Chen

llms agents reinforcement-learning

arXiv AI

Oct 15

Does self-distillation improve reasoning?

arXiv:2610.18613v1 Announce Type: cross Abstract: Self-distillation is an emerging approach that trains a model to imitate its own predictions when conditioned on privileged information, such as corr...

By Jason Ken Adhinarta, Raina Wu, Ashutosh Sharma, Avinash Balakrishnan, Rameswar Panda, Polina Golland, Yoon Kim, Zhang-Wei Hong

agents reinforcement-learning efficiency safety

arXiv AI

Oct 15

Understanding and Reducing Misalignment in Persuasion Optimization

arXiv:2610.16070v1 Announce Type: new Abstract: Frontier models are now trained for complex multi-agent tasks, from running a company to engaging in negotiations. These tasks necessitate training mod...

By Boyi Wei, Ruta Desai, Peter Henderson, Wenyue Hua

agents reinforcement-learning safety

arXiv Computation and Language

Oct 15

Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation

arXiv:2610.19528v1 Announce Type: new Abstract: Retrieving external knowledge can improve large language model (LLM) translation by providing supplementary information such as idiom interpretations a...

By Shijun Li, Yang Liu, Najmeh Sadoughi, Dongmei Wang, Xinjie Li, Zhu Liu, Rohith Mysore Vijaya Kumar, Yuzong Liu

llms rag agents nlp reinforcement-learning

arXiv Machine Learning

Oct 15

Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?

arXiv:2610.19145v1 Announce Type: new Abstract: On-policy distillation (OPD) is increasingly studied as a cheaper alternative to reinforcement learning, yet it remains a form of knowledge distillatio...

By Yiping Li, Fan Zhao, Zhibo Hou, Zhiyu An, Wan Du

reinforcement-learning efficiency

arXiv AI

Oct 15

Discrete Flow Matching for Offline-to-Online Reinforcement Learning

arXiv:2605.12379v2 Announce Type: replace-cross Abstract: Many reinforcement learning (RL) tasks have discrete action spaces, but most generative policy methods based on diffusion and flow matching a...

By Fairoz Nower Khan, Nabuat Zaman Nahim, Peizhong Ju

diffusion reinforcement-learning fine-tuning

arXiv AI

Oct 15

When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL

arXiv:2605.28918v2 Announce Type: replace-cross Abstract: For sparse, structured reinforcement-learning tasks with semantic reward-function interfaces, LLM-generated reward shaping is better framed a...

By Youting Wang, Yuan Tang, Bowen Liu, Xuan Liu, Dingyan Shang

llms agents reinforcement-learning robotics

arXiv AI

Oct 15

Belief Embedding Tree Search

arXiv:2610.15046v1 Announce Type: new Abstract: Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of action...

By Christopher Solinas, Radovan Halu\v{s}ka, Martin Schmid, Michael Buro, Nathan R. Sturtevant, Michael Bowling

rag agents reinforcement-learning

arXiv AI

Oct 15

Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

arXiv:2610.15141v1 Announce Type: new Abstract: Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Referenc...

By Zixun Lan, Zhenfu Liu, Yi Li, Xinheng Wang, Mingmin Chi, Qibiao Wang, Jie Zhu, Xiaohui Tian, Fei Ma

llms reinforcement-learning

arXiv AI

Oct 15

CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning

arXiv:2610.17733v1 Announce Type: cross Abstract: Rubric-based reinforcement learning (RL) evaluates responses against explicit criteria to construct rewards, offering a viable way to train large lan...

By Guangcheng Zhu, Yingfan Ma, Xing Zheng, Haobo Wang, Shenzhi Yang, Kai Tang, Zhengqing Zang, Bowen Song, Weiqiang Wang, Gang Chen

llms reinforcement-learning efficiency benchmarks

arXiv AI

Oct 15

XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learning

arXiv:2610.16478v1 Announce Type: new Abstract: Deep reinforcement learning has shown strong potential for vision-based autonomous driving, but policies trained in simulation often suffer severe perf...

By Wei Yao, Chuyao Wang, Ziwei Wang, Nabil Aouf

agents reinforcement-learning robotics fine-tuning

arXiv AI

Oct 15

EAU: Environment-Agent-User Co-evolution for Training User-Centric Agents

arXiv:2610.16523v1 Announce Type: new Abstract: Training LLM agents to interact with users over multiple turns is challenging because the learning signal depends not only on the agent, but also on th...

By Min Cai, Dan Zhang, Gao Xing, Libin Ma, Zhaohui Xu, Xi Ye

llms agents reinforcement-learning benchmarks

arXiv AI

Oct 15

WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training

arXiv:2610.16152v1 Announce Type: cross Abstract: Although post-training significantly enhances model performance, the use of massive undisclosed data raises concerns about benchmark contamination an...

By Hengxiang Zhang, Junxian Liu, Hongxin Wei

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Oct 15

Cowork-Gym: Synthesizing Interactive Work Environments at Scale

arXiv:2610.17670v1 Announce Type: new Abstract: Cowork agents navigate dependencies across tools and files while adapting to changing user requirements and edits to shared workspaces. Scaling such en...

By Anni Zou, Cheng Chen, Jiaji Deng, Weikang Zhou, Xinji Mai, Yue Wu, Jianing Lin, Lipeng Xie, Lixiong Qin, Qingyue Zhao, Que Shen, Shuchang Tao, Xuchen Pan, Yaliang Li, Yunpeng Zhai, Yushuo Chen, Zhihai Wang, Zhuo Zhang, Zhaoyang Liu, Bolin Ding, Jingren Zhou

agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Oct 15

GameSmith: Scaling Zero-Sum Self-Play with Synthesised Games

arXiv:2610.13995v1 Announce Type: cross Abstract: Post-training large language models (LLMs) on two-player zero-sum games provides a verifiable reward, and the gains have been shown to transfer to ge...

By Weixian Waylon Li, Jiaxin Zhang, Han Zhou, Chengzu Li, Ke Wang, Fengxiang He, Yiwen Guo

llms reinforcement-learning efficiency benchmarks

Get the daily briefing