arXiv Computer Vision

Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation

The paper proposes a method for distilling a vision‑based policy from a state‑based teacher policy in laparoscopic robotic manipulation. By using privileged state observations as a teacher, the vision policy learns from RGB images and proprioception, requiring fewer expert demonstrations than imitation learning and outperforming pure reinforcement learning. Experiments in simulation show that this distillation approach yields more consistent and stable performance across various surgical tasks.

arXiv AI
Jul 7

SutureFormer: Learning Surgical Trajectories via Goal-conditioned Offline RL in Pixel Space

arXiv:2603. 26720v3 Announce Type: replace-cross Abstract: Predicting surgical needle trajectories from endoscopic video is critical for robot-assisted suturing, enabling anticipatory planning, real-time guidance, and safer motion execution.

By Huanrong Liu, Chunlin Tian, Tongyu Jia, Tailai Zhou, Qin Liu, Yu Gao, Yutong Ban, Yun Gu, Guy Rosman, Xin Ma, Qingbiao Li
arXiv Machine Learning
Sep 7

Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics

Squint is a visual Soft Actor Critic algorithm designed to accelerate reinforcement learning for robotics. It combines parallel simulation, a distributional critic, resolution squinting, layer normalization, a tuned update-to-data ratio, and an optimized implementation to reduce wall‑clock training time. On the SO‑101 Task Set, Squint trains policies in as little as 15 minutes on a single RTX 3090 GPU, with most tasks converging in under 6 minutes and successfully transferring to a real SO‑101 robot.

By Abdulaziz Almuzairee, Henrik I. Christensen
arXiv Machine Learning
Jun 2

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

arXiv:2606. 02194v1 Announce Type: new Abstract: Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robotic control, particularly for dexterous manipulation.

By Christian Scherer, Joe Watson, Theo Gruner, Daniel Palenicek, Ingmar Posner, Jan Peters
Hugging Face Trending Papers
Jul 27

DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning

Generalizable robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions. While recent Vision-Language-Action models benefit from large-scale pretraining, their predominantly static pretraining objectives provide limited supervision for physical dynamics and temporal causality, leaving control-relevant knowledge to be learned from downstream robot demonstrations.

arXiv AI
Oct 12

OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models

OGPO is an Offline Goal‑conditioned Policy Optimization framework designed to improve Vision‑Language‑Action (VLA) models for robotic manipulation. It replaces traditional step‑wise action imitation with adaptive relabeling of trajectories using semantic key states and goal‑conditioned rewards, enabling policies to reach task‑progress states even from off‑demonstration states. Experiments on LIBERO, MetaWorld, LIBERO‑Plus, and the proposed LIBERO‑ReAct benchmark show that OGPO consistently outperforms standard supervised fine‑tuning in terms of robustness and recoverability under distribution shifts.

By Xule Gao, Xi Wang, Zehua Zang, Rui Wang, Changwen Zheng, Chuxiong Sun
arXiv Computer Vision
Aug 25

From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation

arXiv:2605.08712v2 Announce Type: replace Abstract: Action-conditioned surgical video generation is a critical yet highly challenging problem for robotic surgery. The core difficulty is that low-dime...

By Bohan Li, Shuojue Yang, Baorui Peng, Xianda Guo, Erli Zhang, Youqi Tao, Junfeng Duan, Daguang Xu, Qi Dou, Xin Jin, Wenjun Zeng, Hao Zhao, Yueming Jin