PiHand is a physics‑based framework that learns latent hand‑object interaction priors for dexterous manipulation. It first trains expert policies to track motion capture demonstrations, then distills these into a goal‑conditioned VAE that captures coupled hand‑object motion. Using bounded residual exploration in this latent space, PiHand produces general tabletop manipulation policies that generalize to unseen objects and trajectories, outperforming baselines across multiple tasks.
By Niklas Berndt, Xianghui Xie, Enes Duran, Yixiao Wang, Gerard Pons-Moll
arXiv:2608.22449v1 Announce Type: cross
Abstract: Forecasting dexterous hand motions from egocentric observations is fundamental to intelligent interactive systems. Existing VLM-based methods typical...
By Wen Wang, Ruibing Hou, Hong Chang, Shiguang Shan, Xilin Chen
AdaDex is a goal‑conditioned reinforcement learning framework that transforms a short hand‑object trajectory—captured via mocap or in‑the‑wild RGB(D) video—into an adaptable dexterous skill. It trains a shared policy on two parallel environment populations: motion‑tracking environments that embed human priors and generalization environments that enforce robustness to noisy states and goals. The resulting policies achieve diverse, functional dexterous behaviors such as grasping, re‑posing, in‑hand reorientation, and dynamic throw‑and‑catch, outperforming baselines in generalization and demonstrating transfer from simulation to the real world.
By Yuxuan Kuang, Jiashun Wang, Yixuan Sun, Yufei Ye, Sungjae Park, Umar Iqbal, Katerina Fragkiadaki, Shubham Tulsiani
Retargeting human object interaction demonstrations to physics based simulation requires reproducing not only body motion but also the object motion and contacts that make manipulation succeed. However, position only hand trajectories do not specify the contact forces needed to manipulate objects, and directly tracking them can overconstrain contact rich finger behavior.
arXiv:2606. 10614v1 Announce Type: cross Abstract: Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots.
By Beomjun Kim, Seong Hyeon Park, Seunghoon Sim, Seungjun Moon, Sanghyeok Lee, Jinwoo Shin
arXiv:2610.14691v2 Announce Type: cross
Abstract: Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos....
By Zihan Gao, Yuhong Zhang, Mengzhe Wang, Kaisheng Liu, Shengpeng Li, Siyao Xiao, Runqing Cheng, Xiao Lin, Zhifang Liu, Haoqian Wang
arXiv:2607. 00033v1 Announce Type: cross Abstract: Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging.
By Xinghao Zhu, Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, Michael Andres Lin, Wei Liu, Tingwu Wang, Xingye Da, Zhengyi Luo, Vishal Kulkarni, Naema Bhatti, Yuke Zhu, Linxi Fan, Bowen Wen, Danfei Xu, Soha Pouya, Yan Chang
Learning humanoid-object interaction requires coordinating whole-body balance, locomotion, and dexterous hand contact to control both robot and object motion. Human demonstrations provide examples of...
arXiv:2608. 19759v1 Announce Type: cross Abstract: Multifingered grasping is a crucial robotic skill, but current deep-learning grasp planners often struggle to generalize to new objects because they are trained on limited, object-specific datasets.
By Julien Merand, Boris Meden, Mathieu Grossard, Liming Chen
arXiv:2610.14372v1 Announce Type: cross
Abstract: Dexterous robot hands can receive useful motion intent from teleoperation or learned policies, but successful execution also depends on contact decis...
By Weikang Wan, Zeyu Lai, Jianyu Hou, Yuchen Zhou, Jiawei Fu, Henrik I Christensen, Hao Su
DexPort is a portable interface that lets operators collect high‑quality dexterous hand demonstrations without a full robot setup. By using the same hand hardware for both collection and deployment, it records joint targets and states while allowing operators to specify wrist motion and select from a small set of coordinated finger skills. The authors collected over 2,500 demonstrations across four real‑world tasks, showing significant performance gains when using recorded robot commands and in‑the‑wild data.
By Amber Xie, Jiawei Fu, Xiaofeng Guo, Carmelo Sferrazza, Guanya Shi, Haozhi Qi
The paper introduces DSP: Dexterity from Supervised Play, a framework where a robot learns a play policy from brief human demonstrations via behavior cloning and then autonomously gathers extensive interaction data to train a world model. During inference, the play policy guides model‑predictive control by proposing action sequences evaluated by the learned model, and a self‑play distillation step produces a closed‑loop, goal‑conditioned policy. Experiments on underactuated pushing and in‑hand pose reorientation show DSP outperforms hindsight behavior cloning by 69.6% in success rate and matches reinforcement learning performance while using 490 times less data.