← Back to all news
arXiv Machine Learning By Alexander Swerdlow, Perry Dong, Chelsea Finn

Generative Value Classifier

Read the original on arXiv Machine Learning →

The paper introduces a generative value classifier that rethinks Q‑learning by predicting the likelihood of an action given a return, rather than predicting returns directly. This approach uses a return‑conditioned policy as a generative component, allowing the critic to be supervised by every action and anchored with a squared‑error loss. Experiments on offline and online reinforcement learning tasks demonstrate improvements over leading discriminative and distributional RL methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 21

Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR

arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.

By Jiaming Li, Longze Chen, Ze Gong, Yukun Chen, Lu Wang, Wanwei He, Run Luo, Min Yang
llmsreinforcement-learning
More like this →
arXiv Machine Learning
Sep 29

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.

By Yichen Lin, Xuyuan Xiong, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao
llmsreinforcement-learningbenchmarksdata-engineering
More like this →