arXiv Machine Learning September 29, 2026 By Yichen Lin, Xuyuan Xiong, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 18

Q-Regularized Generative Auto-Bidding: From Suboptimal Trajectories to Optimal Policies

arXiv:2601. 02754v3 Announce Type: replace-cross Abstract: With the rapid development of e-commerce, auto-bidding has become a key asset in optimizing advertising performance under diverse advertiser environments.

By Mingming Zhang, Na Li, Zhuang Feiqing, Hongyang Zheng, Jiangbing Zhou, Wang Wuyin, Sheng-jie Sun, XiaoWei Chen, Junxiong Zhu, Lixin Zou, Chenliang Li

llms reinforcement-learning benchmarks

Hugging Face Trending Papers

3d ago

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning

The paper introduces Q-learning Penalized Transformer (QPT), a training–inference consistent framework for safe offline reinforcement learning. QPT trains a Transformer policy that generates actions conditioned on trajectory context and target return/cost while incorporating a Q-shaped penalty to balance safety, reward maximization, and behavior regularization. The method consistently outperforms strong baselines on 38 DSRL benchmark tasks and adapts robustly to varying constraint thresholds.

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 18

Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning

arXiv:2608. 15088v1 Announce Type: cross Abstract: Human-in-the-loop (HIL) online reinforcement learning for real robots must absorb human interventions quickly while continuing to improve beyond the human prior.

By Zihang Wang, Yishan Wang

agents reinforcement-learning robotics

arXiv Machine Learning

2d ago

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning

arXiv:2609.34426v2 Announce Type: replace Abstract: This paper addresses the problem of safe offline reinforcement learning, which involves training a policy to satisfy safety constraints using an of...

By Shengchao Hu, Peng Wang, Jifeng Hu, Qiyang Zhou, Anning Hu, Li Shen, Ya Zhang, Dacheng Tao

llms reinforcement-learning benchmarks safety

Hugging Face Trending Papers

Jul 29

Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?

Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) this raises a natural question: given a pretrained policy, should the Q-function be pretrained on offline data too? Conventional wisdom suggests it should, but recent results show that online RL with a randomly-initialized Q-function can result in highly performant and reliable policies without needing to pretrain the Q-function.

reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 10

Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning

arXiv:2606. 11087v1 Announce Type: cross Abstract: Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control.

By Zhiyuan Zhou, Andy Peng, Charles Xu, Qiyang Li, Tobias Springenberg, Kevin Frans, Sergey Levine

diffusion reinforcement-learning robotics benchmarks

One email a morning, machine-written

Related stories

Q-Regularized Generative Auto-Bidding: From Suboptimal Trajectories to Optimal Policies

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning

Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning

Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?

Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning