arXiv AI October 26, 2026 By Ruiqi Wang, Dezhong Zhao, Xuesu Xiao, Erdem Biyik, Byung-Cheol Min, Guohua Chen

PrefO2O: Zero-Shot Preference-Based Reinforcement Learning via Offline-to-Online Latent Preference Transfer

PrefO2O introduces an offline-to-online latent preference transfer framework that enables zero‑shot preference‑based reinforcement learning on unseen tasks without human feedback. It learns a task‑invariant latent preference space from limited offline preference data, disentangling transferable preference structures from task‑specific variations. The method uses confidence‑weighted optimal transport to generate reliable pseudo‑preference supervision for online reward adaptation, achieving performance that matches or exceeds traditional PbRL methods trained with real human feedback across various robotic manipulation and locomotion benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 2

From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning

arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.

By Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen, Ping-Chun Hsieh

reinforcement-learning fine-tuning

arXiv AI

Jul 1

Freeform Preference Learning for Robotic Manipulation

arXiv:2606. 32027v1 Announce Type: cross Abstract: Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal.

By Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn

agents computer-vision reinforcement-learning robotics safety

arXiv AI

Oct 16

LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation

LIBERO-PeRM is a benchmark for studying personalized robotic manipulation, focusing on how policies learn and follow user preferences in zero‑shot settings, from demonstrations, and across varying tasks and layouts. It introduces a taxonomy of language‑expressible preferences—motion, skill, and user‑profile choices—within the LIBERO simulation framework, generating 312 tasks with paired demonstrations and preference‑satisfaction metrics. Experiments reveal that current vision‑language‑action policies struggle with zero‑shot personalization, while supervised fine‑tuning improves learnability, yet challenges remain in generalization, decomposition, and conflict handling.

By Zhixu Li, Keqian Tang, Litian Gong, Jingyu Yao, Wenqian Zhang, Tianze Xu, Zehao Wang, Yuping Wang, Junge Zhang, Jiachen Li

agents robotics fine-tuning benchmarks

arXiv Machine Learning

Jun 11

PAWS: Preference Learning with Advantage-Weighted Segments

arXiv:2606. 11982v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) learns policies from human trajectory-level comparisons, avoiding explicit reward design and expert demonstrations.

By Aleksandar Taranovic, Onur Celik, Niklas Freymuth, Ge Li, Serge Thilges, Huy Le, Tai Hoang, Rania Rayyes, Gerhard Neumann

reinforcement-learning robotics

arXiv Machine Learning

Aug 13

Adaptation of Generalist Robot Policies with Minimal Data

arXiv:2608. 11363v1 Announce Type: cross Abstract: A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction.

By Shreyas Kowshik, Sreyas Venkataraman, Leo Wang, Niharika Pant, Max Simchowitz, Aviral Kumar

agents reinforcement-learning robotics

arXiv Machine Learning

Jul 9

Latent Policy Steering through One-Step Flow Policies

arXiv:2603. 05296v2 Announce Type: replace-cross Abstract: Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration.

By Hokyun Im, Andrey Kolobov, Jianlong Fu, Youngwoon Lee

reinforcement-learning robotics efficiency benchmarks

One email a morning, machine-written

Related stories

From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning

Freeform Preference Learning for Robotic Manipulation

LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation

PAWS: Preference Learning with Advantage-Weighted Segments

Adaptation of Generalist Robot Policies with Minimal Data

Latent Policy Steering through One-Step Flow Policies