Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

3,495 stories · RSS feed

arXiv Machine Learning

Aug 5

LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation

arXiv:2608. 03382v1 Announce Type: cross Abstract: Multi-armed bandit algorithms, especially Thompson sampling, are widely used in online recommendation.

By Eugene Lee, Oseong Choi, Byungsoo Kang, Taeyeong Jang

llms reinforcement-learning safety

arXiv Machine Learning

Aug 5

Dynamically Allocating Evaluation Effort for Model Ranking

arXiv:2608. 03437v1 Announce Type: cross Abstract: While human evaluation is the gold standard in many NLP tasks, it suffers from prohibitive costs and poor scalability.

By Vil\'em Zouhar, Julia Kreutzer, Alon Lavie, Tom Kocmi, Matt Post, Ond\v{r}ej Bojar, Mrinmaya Sachan

nlp reinforcement-learning benchmarks

arXiv AI

Aug 5

Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution

arXiv:2608. 03483v1 Announce Type: cross Abstract: Existing chunk-based Vision-Language-Action (VLA) models execute a fixed number of actions (i.

By Weichen Xu, Zhenhua Liu, Lin Luo, Yaobo Liang, Chengtang Yao, Qingyu Mei, Jian Cao, Xixin Cao, Xing Zhang, Jiaolong Yang, Baining Guo

reinforcement-learning robotics multimodal safety

arXiv AI

Aug 5

In-Context Pure Exploration in Continuous Decision Spaces

arXiv:2602. 17976v2 Announce Type: replace-cross Abstract: In active sequential testing, also termed pure exploration, a learner is tasked with the goal to adaptively acquire information so as to identify an unknown ground-truth hypothesis with as few queries as possible.

By Alessio Russo, Yin-Ching Lee, Ryan Welch, Aldo Pacchiano

reinforcement-learning

arXiv Machine Learning

Aug 5

DiagLoop: A Counterfactual Data Flywheel with Stage-Localized Reinforcement for Diagnostic LLMs

arXiv:2608. 03674v1 Announce Type: new Abstract: Causal diagnostic models must explain how conclusions follow from evidence because diagnoses guide repairs and treatments.

By Jian Zhang, Bingyi Wang, Yizhi Liu

llms reinforcement-learning

arXiv AI

Aug 5

PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

arXiv:2608. 03682v1 Announce Type: new Abstract: Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, and onboard deployment.

By Chenghua Wang, Daliang Xu, Dongqi Cai, Duojin Sun, Hao Zhang, Haoze Qian, Huaiyuan Zhang, Jinshuo Cui, Kezhao Zhao, Longxi Gao, Mengwei Xu, Rongjie Yi, Tianyue Zhang, Weikai Xie, Xiyuan Tan, Xuanzhe Liu, Yingying Qin, Yiwen Lu, Yuan Yao, Yuezhi Zu, Yunhan Guo, Ziqi Guo

reinforcement-learning robotics efficiency multimodal benchmarks

arXiv AI

Aug 5

Training Documents Reranker with Search Rubrics for Deep Research Agent

arXiv:2608. 03527v1 Announce Type: cross Abstract: Retrieval systems help deep research agents generate high-quality answers by providing relevant documents.

By Wenhan Liu, Yu Lu, Qiaolin Xia, Hui Xu, Tong Zhao, Jian Xi, Yutao Zhu, Haijin Liang, Haibo Shi, Hao Wang, Zhicheng Dou

llms rag agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 5

SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

arXiv:2607. 27703v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning.

By Yang Zhou, Zixuan Huang, Sunzhu Li, Zhuo Yang, Chen Zhang, Shunian Chen, Caijun Yan, Jianyao Xu, Shunyu Liu, Weijie Fu, Peiliang Li, Xiaozhi Chen, Yuxiang Cai

llms agents computer-vision reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Aug 5

When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs

arXiv:2608. 03506v1 Announce Type: new Abstract: Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confounding error, and votes fragment across multiple valid answers, letting an invalid answer win despite a valid minority trace.

By Omatharv Bharat Vaidya, Connor Thomas Jerzak, Zayne Rea Sprague, Fangcong Yin, Nhat Ho

llms reinforcement-learning benchmarks

arXiv AI

Aug 5

Learning Clinical-Trial Strategy: Offline Policy Training for Decision Agents

arXiv:2608. 03606v1 Announce Type: new Abstract: Clinical development is sequential decision-making under uncertainty, where a sponsor must plan a portfolio of experiments from heterogeneous evidence.

By William Bolton, Philip Torr

llms agents reinforcement-learning fine-tuning

arXiv AI

Aug 5

Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR

arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.

By Yongshi Ye, Liang Zhang, Yidong Chen, Xiaodong Shi, Biao Fu

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 5

Latent Reward Registers for Diffusion Preference Alignment

arXiv:2608. 03929v1 Announce Type: new Abstract: Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment challenge across the multi-step denoising process.

By Yuanshen Guan, Zipeng Feng, Zhiwei Xiong, Peiqin Sun

llms diffusion reinforcement-learning efficiency benchmarks safety

arXiv AI

Aug 5

Target-Aligned Fusion for Decision-Sequence Learning under Dynamics Shift

arXiv:2511. 09173v3 Announce Type: replace-cross Abstract: External trajectories can improve offline decision-sequence learning, but dynamics shift may make some source subsequences inconsistent with the target environment.

By Guojian Wang, Quinson Hon, Xuyang Chen, Lin Zhao

llms reinforcement-learning safety

arXiv AI

Aug 5

ToolLIFT: Lifting Tool-Specific Trajectories into Function-Level Graphs for Generalizable Tool Planning

arXiv:2608. 03468v1 Announce Type: new Abstract: Historical tool-use trajectories provide valuable experience for large language model (LLM) agents to plan and coordinate tool usage.

By Xiuhui You, Jiayi Luo, Zichao Shen, Qingyun Sun, Ziwei Zhang

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 5

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

arXiv:2608. 03036v1 Announce Type: cross Abstract: Large Language Models (LLMs) are integrated into software systems and AI services, making efficient LLM serving a concern for software engineering.

By Forough Majidi, Mohammad Mehdi Morovati, Foutse Khomh, Heng Li

llms reinforcement-learning efficiency multimodal

arXiv AI

Aug 5

Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning

arXiv:2608. 02993v1 Announce Type: new Abstract: (Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning.

By Subrat Prasad Panda, Blaise Genest, Arvind Easwaran

agents reinforcement-learning

arXiv AI

Aug 5

CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting

arXiv:2608. 03031v1 Announce Type: new Abstract: Time series forecasting is fundamental to decision-making in complex systems, where future dynamics are influenced not only by historical observations but also by evolving contextual features.

By Xiaoyu Tao, Mingyue Cheng, Bokai Pan, Chuang Jiang, Huanjian Zhang, Tian Gao, Yaguo Liu, Qi Liu, Enhong Chen

llms agents reinforcement-learning fine-tuning

arXiv AI

Aug 5

Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning

arXiv:2608. 03875v1 Announce Type: cross Abstract: Designing effective reward functions remains a major bottleneck in Reinforcement Learning (RL).

By Pyrros Koussios, Chenhao Li, Xin Chen, Andreas Krause

llms diffusion reinforcement-learning fine-tuning multimodal safety

arXiv Machine Learning

Aug 5

Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning

arXiv:2508. 20697v4 Announce Type: replace Abstract: As large language models (LLMs) continue to grow in capability, so do the risks of harmful misuse through fine-tuning.

By Weitao Feng, Lixu Wang, Peizhuo Lv, Tianyi Wei, Jie Zhang, Chongyang Gao, Sinong Zhan, Wei Dong

llms reinforcement-learning fine-tuning safety

arXiv AI

Aug 5

SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation

arXiv:2608. 03092v1 Announce Type: cross Abstract: We aim to improve model performance in multi-reward reinforcement learning training process.

By Wen Wang, Jiahua Bao, Tu Yongsiqi, Yihao Liu, Haotian Zhou, Haoxuan Ma, Mengyu Zhou, Wenkui Fan, Junwei He, Xiaoxi Jiang, Guanjun Jiang

reinforcement-learning efficiency