AdaDex is a goal‑conditioned reinforcement learning framework that transforms a short hand‑object trajectory—captured via mocap or in‑the‑wild RGB(D) video—into an adaptable dexterous skill. It trains a shared policy on two parallel environment populations: motion‑tracking environments that embed human priors and generalization environments that enforce robustness to noisy states and goals. The resulting policies achieve diverse, functional dexterous behaviors such as grasping, re‑posing, in‑hand reorientation, and dynamic throw‑and‑catch, outperforming baselines in generalization and demonstrating transfer from simulation to the real world.
arXiv:2606. 10614v1 Announce Type: cross Abstract: Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots.
arXiv:2610.14691v2 Announce Type: cross Abstract: Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos....
Dex-One2Many introduces a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. By abstracting the video into sequential scene graphs, the method guides reinforcement learning with dense rewards and diverse reset states, enabling efficient exploration while preserving broad generalizability. Trained entirely in simulation, Dex-One2Many achieves zero‑shot transfer to a real multi‑fingered hand and outperforms baselines by 6.5% on seen configurations and 71% on unseen scenarios across five tool‑use and manipulation tasks.
ADEPT is a reinforcement‑learning framework that first pre‑trains a dexterous policy on a generic object reposing task and then post‑trains downstream policies using this pretrained behavior as a prior. The approach avoids relearning basic skills for each new task, and employs a stable post‑training recipe—behavior‑cloning distillation, critic warm‑up, and conservative on‑policy updates—to preserve the pretrained capabilities. ADEPT’s joint‑space Geometric Fabric mediates between the policy and the robot, enabling zero‑shot sim‑to‑real transfer on a 23‑DoF Kuka‑Allegro and a 29‑DoF Flexiv‑Sharpa, where the robots solve long‑horizon tasks from challenging initial states at human‑level speed.
arXiv:2608. 15917v1 Announce Type: cross Abstract: Large-scale pre-training has made robot policy fine-tuning increasingly data-efficient, but this progress has largely been driven by datasets and embodiments built around simple parallel-jaw grippers.
arXiv:2607. 11874v1 Announce Type: cross Abstract: Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion to robot kinematic references, then train policies via reinforcement learning (RL) to track them.
Humanoid loco-manipulation is often simplified into a stop-and-go process: walking to an object, stopping to manipulate it, and then resuming locomotion. It also commonly relies on low degree-of-freedom (DoF) end effectors that behave like an open-close grasp primitive.
Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion to robot kinematic references, then train policies via reinforcement learning (RL) to track them. But how does this recipe transfer to dexterous manipulation?
The paper introduces an interaction‑centric framework that unifies representations for two‑finger gripper manipulation across different robot embodiments. By using a parameterized universal gripper abstraction and a canonical gripper‑frame representation, the system infers sub‑tasks from language and RGB‑D inputs, grounds interaction triplets, and employs hybrid features and a Flow‑Matching Transformer to generate smooth 7‑DoF action sequences. Experiments in both simulation and real‑world settings show that this approach achieves competitive benchmark performance while enabling extreme cross‑embodiment and cross‑viewpoint zero‑shot sim‑to‑real transfer to heterogeneous robot platforms.
arXiv:2606. 08057v1 Announce Type: cross Abstract: Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets.
DexMimic is a generalist dexterous hand‑object tracking policy trained across diverse human hand‑object interaction trajectories. It uses Value‑Guided Reset Allocation to adapt the initialization distribution, prioritizing difficult phases and selecting reset states that maximize expected tracking returns. The authors present a benchmark for training, fine‑tuning, and generalization, demonstrating improved tracking success, training efficiency, faster fine‑tuning, and potential zero‑shot generalization to unseen trajectories and objects, and validate the approach on a real‑world dexterous manipulation platform.
EgoJEPA is a three‑stage framework that learns embodiment‑agnostic representations from egocentric human videos, eliminating the need for hand‑crafted retargeting mappings. It first pre‑trains latent actions via visual prediction conditioned on hand‑motion differences, then models future visual dynamics and latent actions jointly, and finally adapts to specific robots with lightweight action adapters. Experiments on five manipulation tasks across three robot embodiments show a 74% average success rate versus 65% for a retargeting baseline, while cutting cross‑embodiment wall‑clock cost by 47.2%.