← Back to all news
arXiv Machine Learning

From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation

The study evaluated 281 imitation‑learning policies across seven families in 1,700 real‑robot trials of RJ45 connector grasping and insertion with a five‑fingered hand. Controlled ablations revealed that observation and action representations—such as ROI cropping, robot‑base framing, and point‑encoder normalization for point‑cloud policies, and absolute action prediction for vision‑language‑action policies—drastically improved success rates. Among 67 offline metrics, only five measures of predicted‑motion smoothness and directness correlated reliably with real‑world success, indicating that offline metrics can help shortlist policies but cannot replace actual robot testing.

Related stories

arXiv Computer Vision
6d ago

BiDex-100: A Comprehensive Benchmark for Bimanual Dexterous Manipulation Across 100+ Simulation Tasks

BiDex-100 is a simulation benchmark for bimanual dexterous manipulation, featuring 106 tasks across five capability axes—generalization, instruction following, limited demonstrations, long horizon, and memory. Each task includes an out‑of‑domain variant to test robustness, and the benchmark supports two hand embodiments with 8,080 teleoperated demonstrations totaling 6.25 million frames. The authors evaluate seven recent Vision‑Language‑Action models under a unified protocol, providing baseline performance for capability, robustness, and data efficiency.

By Yong Wang, Kecheng Zheng, Xing Zhu, Chao Xu, Yujun Shen, Shengze Cai
roboticsmultimodalbenchmarks
More like this →
arXiv AI
Jun 9

BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models

arXiv:2605. 30226v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for grounding visual-language understanding into real-world robotic manipulation.

By Zhongxi Chen, Yifan Han, Yanming Shao, Huanming Liu, Congsheng Xu, Xiaoyu Chen, Yao Mu, Wenzhao Lian
reinforcement-learningroboticsmultimodal
More like this →
arXiv Computer Vision
Oct 16

EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos

EgoJEPA is a three‑stage framework that learns embodiment‑agnostic representations from egocentric human videos, eliminating the need for hand‑crafted retargeting mappings. It first pre‑trains latent actions via visual prediction conditioned on hand‑motion differences, then models future visual dynamics and latent actions jointly, and finally adapts to specific robots with lightweight action adapters. Experiments on five manipulation tasks across three robot embodiments show a 74% average success rate versus 65% for a retargeting baseline, while cutting cross‑embodiment wall‑clock cost by 47.2%.

By Meng Cao, Can Zhao, Jinjie Wei, Haonan Zhao, Yulong Zhang, Duo Han, Xiaofan Li, Ziwu Song, Rongtao Xu, Ruyi Gan, Hao Wang, Qian Wang, Daolin Ma, Ian Reid, Xiaodan Liang
robotics
More like this →
arXiv Machine Learning
Jun 10

Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations

arXiv:2606. 10614v1 Announce Type: cross Abstract: Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots.

By Beomjun Kim, Seong Hyeon Park, Seunghoon Sim, Seungjun Moon, Sanghyeok Lee, Jinwoo Shin
llmsagentsroboticsfine-tuningbenchmarks
More like this →
arXiv AI
Sep 24

BEE: Intervention-Adaptive Real-World Reinforcement Learning with Vision-Language-Action Models

arXiv:2609.27450v1 Announce Type: cross Abstract: Vision-language-action (VLA) models handle long-horizon manipulation, yet success hinges on a few precision-critical phases where millimeter-scale er...

By Weihui Zhao, Xiaohan Yan, Zunian Wan, Xuan Du, Zhaozhan Chi, Jianbo Mao, Ruipu Wu, Rushuai Yang, Houlin Li, Shukai Yang, Jing Wu, Yuxiang Yan, Yongcheng Liu, Chuankang Li, Guanghui Ren, Wei Shan, Maoqing Yao
reinforcement-learningroboticsmultimodal
More like this →
arXiv AI
Oct 26

DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection

DexPort is a portable interface that lets operators collect high‑quality dexterous hand demonstrations without a full robot setup. By using the same hand hardware for both collection and deployment, it records joint targets and states while allowing operators to specify wrist motion and select from a small set of coordinated finger skills. The authors collected over 2,500 demonstrations across four real‑world tasks, showing significant performance gains when using recorded robot commands and in‑the‑wild data.

By Amber Xie, Jiawei Fu, Xiaofeng Guo, Carmelo Sferrazza, Guanya Shi, Haozhi Qi
agentsrobotics
More like this →
arXiv AI
Jul 29

InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulation

arXiv:2606. 12109v2 Announce Type: replace-cross Abstract: Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel-gripper action interfaces do not specify how those priors should be realized by a dexterous hand.

By Chuanke Pang, Junyi Huang, Zhijun Zhao, Yaobing Wang, Kun Xu, Xilun Ding
diffusionroboticsmultimodal
More like this →