arXiv AI

AC-ODM: Actor--Critic Online Data Mixing for Sample-Efficient LLM Pretraining

arXiv:2505. 23878v2 Announce Type: replace-cross Abstract: Optimizing pretraining data composition is pivotal for LLM generalization.

arXiv AI

Jul 23

Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models

arXiv:2607. 19399v1 Announce Type: cross Abstract: It is commonly observed that online reinforcement learning (RL) produces better-performing strategies than offline methods across a broad range of performance measures.

By Dmitriy Poyarkov, Aleksei Staroverov, Aleksandr I. Panov

reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jul 14

Active Offline-to-Online Reinforcement Learning

arXiv:2607. 11720v1 Announce Type: cross Abstract: Background: Offline reinforcement learning (RL) enables effective policies to be trained from large, previously collected datasets and subsequently improved through limited online interaction.

By Alper Kamil Bozkurt, Shangtong Zhang, Yuichi Motai

reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 24

Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning

arXiv:2606. 24133v1 Announce Type: new Abstract: The composition of training data, governed by the diversity of sources and their mixing strategy, is a cornerstone of Large Language Model (LLM) pre-training.

By Chenhao Dang, Jing Ma, Mingjie Liao

llms reinforcement-learning benchmarks

arXiv AI

Aug 12

Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning

arXiv:2608. 10473v1 Announce Type: cross Abstract: Offline-to-online (O2O) reinforcement learning aims to leverage policies pretrained on static datasets while improving them through online interaction.

By Daoyi Li, Yixian Zhang, Chao Yu, Wenbo Ding, Yu Wang

reinforcement-learning fine-tuning

Hugging Face Trending Papers

Jul 13

Active Offline-to-Online Reinforcement Learning

Background: Offline reinforcement learning (RL) enables effective policies to be trained from large, previously collected datasets and subsequently improved through limited online interaction. This offline-to-online RL (O2O-RL) paradigm is particularly promising in nonstationary domains where interaction is costly or potentially hazardous.

reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 16

Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning

arXiv:2606. 15333v1 Announce Type: cross Abstract: LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility.

By Zirui Pang, Chenlong Zhang, Haosheng Tan, Zhuoran Jin, Jiaheng Wei, Zixin Zhong

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 4

ReBRAC-v2: The Return of the King

arXiv:2608. 01205v1 Announce Type: new Abstract: Recent offline reinforcement learning methods increasingly rely on expressive generative policies and specialized value-guidance mechanisms.

By Denis Tarasov, Robert K. Katzschmann

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 16

Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data

arXiv:2502. 19544v3 Announce Type: replace Abstract: Leveraging offline data is a promising way to improve the sample efficiency of online reinforcement learning (RL).

By Yi Zhao, Aidan Scannell, Wenshuai Zhao, Yuxin Hou, Tianyu Cui, Le Chen, Dieter B\"uchler, Arno Solin, Juho Kannala, Joni Pajarinen

reinforcement-learning robotics fine-tuning

arXiv AI

Jul 7

R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning

arXiv:2601. 11960v3 Announce Type: replace-cross Abstract: Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide with the one producing inference responses.

By Jingchu Wang, Bingbing Xu, Yige Yuan, Dan Zhang, Bin Xie, Xiaoqian Sun, Huawei Shen

llms reinforcement-learning safety

arXiv Machine Learning

Jun 30

The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning

arXiv:2606. 29526v1 Announce Type: new Abstract: Reinforcement learning (RL) has gained growing attention in large language model (LLM) post-training, yet RL training remains fragile and can suffer from instability or collapse.

By Jing Liang, Hongyao Tang, Yi Ma, Yancheng He, Weixun Wang, Xiaoyang Li, Ju Huang, Wenbo Su, Jinyi Liu, Yan Zheng, Jianye Hao, Bo Zheng

llms reinforcement-learning

arXiv Machine Learning

Jul 21

Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraints

arXiv:2607. 17326v1 Announce Type: new Abstract: Transfer-oriented reinforcement learning requires evaluating algorithms along dimensions that go beyond standard sample efficiency.

By Hany Hamed, Abhishek Naik, Colin Bellinger, A. Rupam Mahmood

reinforcement-learning

arXiv AI

Aug 10

MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents

arXiv:2608. 07068v1 Announce Type: new Abstract: Long-horizon agents accumulate growing contexts during interaction, impairing performance and stability.

By Zhiyuan Liu, Tinghong Ye, Chenghao Liu, Yizhuo Li, Songfang Huang

agents reinforcement-learning efficiency safety

Related stories