arXiv AI July 1, 2026 By Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn

Freeform Preference Learning for Robotic Manipulation

arXiv:2606. 32027v1 Announce Type: cross Abstract: Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 24

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics

arXiv:2602. 19313v2 Announce Type: replace-cross Abstract: General-purpose robot learning requires dense, instruction-conditioned feedback that can distinguish meaningful task progress from stalled, failed, or partially completed behavior.

By Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang, Zhongzheng Ren, Lillian J. Ratliff, Jiafei Duan, Dieter Fox, Ranjay Krishna

llms robotics multimodal benchmarks

arXiv Machine Learning

Aug 11

RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance

arXiv:2608. 09853v1 Announce Type: cross Abstract: General-purpose reward models are increasingly the bottleneck for scaling robot learning, yet the recipe for learning value-related capabilities from large-scale heterogeneous corpora remains underexplored.

By Dongchi Huang, Hongyin Zhang, Bohan Hou, Siteng Huang, Zhian Su, Hang Guo, Tong Lu, Zhaofeng Xu, Jiahao Tang, Jianfei Yang, Donglin Wang, Peixi Peng, Mingxiu Chen, Deli Zhao, Xin Li

robotics

arXiv AI

Jun 2

From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models

arXiv:2606. 00083v1 Announce Type: cross Abstract: Reinforcement learning relies on accurate reward functions, which are often hand-crafted or even unavailable in real-world applications, such as robotics.

By Christian Gumbsch, Leonardo Barcellona, Lennard Sch\"unemann, Platon Karageorgis, Andrii Zadaianchuk, Zehao Wang, Sergey Zakharov, Fabien Despinoy, Rahaf Aljundi, Efstratios Gavves

llms reinforcement-learning robotics multimodal

arXiv AI

Jun 2

SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning

arXiv:2512. 00062v2 Announce Type: replace-cross Abstract: Robotic policy learning for complex real-world manipulation tasks has seen rapid recent progress, enabled in large part by the ability to collect demonstrations through human operation.

By Taewook Nam, Junmo Cho, Youngsoo Jang, Sung Ju Hwang

reinforcement-learning robotics fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 2

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

arXiv:2606. 02194v1 Announce Type: new Abstract: Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robotic control, particularly for dexterous manipulation.

By Christian Scherer, Joe Watson, Theo Gruner, Daniel Palenicek, Ingmar Posner, Jan Peters

reinforcement-learning robotics fine-tuning

arXiv Machine Learning

Jun 11

PAWS: Preference Learning with Advantage-Weighted Segments

arXiv:2606. 11982v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) learns policies from human trajectory-level comparisons, avoiding explicit reward design and expert demonstrations.

By Aleksandar Taranovic, Onur Celik, Niklas Freymuth, Ge Li, Serge Thilges, Huy Le, Tai Hoang, Rania Rayyes, Gerhard Neumann

reinforcement-learning robotics

Related stories

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics

RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance

From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models

SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

PAWS: Preference Learning with Advantage-Weighted Segments