← Back to all news
arXiv Machine Learning By Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar, Guanning Zeng, Qingyang Wu, Zhongzhu Zhou, Chenfeng Xu, Haiwen Feng, Yuda Song, Aarti Singh, Ruslan Salakhutdinov, J. Andrew Bagnell, Jeff Schneider, Andrea Zanette

Tail-Likelihood Reinforcement Learning

Read the original on arXiv Machine Learning →

Tail-Likelihood Reinforcement Learning (TailRL) is a new approach that optimizes the probability of exceeding randomly chosen reward thresholds instead of just the expected reward. By converting continuous rewards into a family of binary success events, TailRL gives more weight to rare, high-reward rollouts, effectively acting as a mixture of Best‑of‑(k) gradients. The method requires only a simple adjustment to the advantage function, making it compatible with existing reinforcement learning pipelines and improving performance across tasks such as object localization, maze navigation, GUI grounding, and code optimization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 21

Maximum Likelihood Reinforcement Learning

arXiv:2602. 02710v2 Announce Type: replace Abstract: Reinforcement learning (RL) is the method of choice for training models in setups where the objective function can only be evaluated by sampling from the model.

By Fahim Tajwar, Guanning Zeng, Yueer Zhou, Yuda Song, Daman Arora, Yiding Jiang, Jeff Schneider, Ruslan Salakhutdinov, Haiwen Feng, Andrea Zanette
reinforcement-learning
More like this →
arXiv Machine Learning
Jul 8

Supervised Reward Inference

arXiv:2502. 18447v2 Announce Type: replace Abstract: Existing approaches to reward inference typically assume that humans provide demonstrations according to specific behavior models.

By Will Schwarzer, Jordan Schneider, Philip S. Thomas, Scott Niekum
roboticsbenchmarks
More like this →
arXiv AI
Aug 28

Residual Reward Models: Leveraging Prior Knowledge for Efficient Preference-based Reinforcement Learning in Robotics

The paper introduces Residual Reward Models (RRM) to enhance preference‑based reinforcement learning (PbRL) in robotics. RRMs decompose the true reward into a prior component—such as a heuristic, language‑generated, or IRL‑derived reward—and a learned residual that is trained with human preferences. Experiments on Meta‑World, DM‑Control, and a physical Franka Panda robot show that RRMs markedly improve sample efficiency and accelerate policy learning compared to standard PbRL methods.

By Chenyang Cao, Miguel Rogel-Garc\'ia, Mohamed Nabail, Xueqian Wang, Nicholas Rhinehart
reinforcement-learningroboticsfine-tuning
More like this →