arXiv AI
Jul 31
From Passive Video to Editable Experience: Physically Grounded Experience Synthesis for Embodied Intelligence
arXiv:2607. 26903v1 Announce Type: new Abstract: The key bottleneck in embodied AI is not model architecture but data.
Human2Any: Human-to-Robot Transfer via Constraint-Aware Compositional Planning
arXiv:2606. 28813v1 Announce Type: cross Abstract: Human videos are a scalable source of supervision for robot manipulation, as they are abundant and naturally capture rich object interactions.
Related stories
arXiv AI
Sep 16
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
HumanEgo is a framework that enables zero‑shot robot learning from short egocentric human videos by converting each demonstration into an entity‑level hand‑object interaction representation and training a flow‑matching policy with dense auxiliary objectives. The method is robot‑data‑free, hardware‑agnostic, and data‑efficient, achieving 92.5 % success on four real‑world tasks with only 30 minutes of human video per task and outperforming matched‑time robot teleoperation by 41 %. HumanEgo also robustly transfers zero‑shot across new robots, cameras, and environments, and is released as an open‑source tool for learning robot policies directly from human data.
Hugging Face Trending Papers
Jul 29
ContactFlow: A video action conditioning that transfers across embodiments
World models offer a promising route toward robot planning by enabling agents to imagine and verify the consequences of actions before execution. However, current video-based world models often struggle to capture the physical constraints that govern manipulation, particularly contact.
arXiv AI
Sep 21
KnowDemo: Knowledge-Guided Robot Demonstration Generation from Human Videos
KnowDemo is a framework that generates diverse robot demonstrations from human videos by leveraging structured manipulation knowledge. It uses a vision‑language model to extract task requirements and permissible execution variations, then resolves these against target‑scene entities to guide candidate generation and screening before motion planning. The resulting demonstrations feature multimodal behavior, alternative contact strategies, and valid subtask orders, and have been shown to improve planning success and enable sim‑to‑real policy transfer across three tasks.
arXiv AI
Jun 2
From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.
arXiv Machine Learning
Sep 17
Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulation
The paper introduces a metric interaction framework for robotic manipulation that explicitly models object- and scene-level interactions in Cartesian space. It uses Interaction‑Centric Tokens (ICTs) to represent end‑effector trajectories relative to objects and a Metric Action Interaction Field (MAIF) to attend to scene point‑cloud features for geometry‑conditioned action corrections. Experiments show modest but consistent improvements across several benchmarks, including LIBERO, RoboTwin 2.0, and real‑world tasks.
arXiv Machine Learning
Jun 10
Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations
arXiv:2606. 10614v1 Announce Type: cross Abstract: Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots.
arXiv Computer Vision
Sep 22
AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
arXiv:2609.22332v1 Announce Type: cross Abstract: Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act...
Hugging Face Trending Papers
Sep 16
Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulation
Vision-Language-Action models and World-Action Models have advanced language-conditioned robotic manipulation, yet often leave metric relations among actions, objects, and scene geometry implicit. Hum...
Hugging Face Trending Papers
5d ago
From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations
The paper introduces P2P‑T, a data‑efficient, object‑centric framework that learns tool manipulation directly from human video demonstrations. It uses a two‑stage approach: first pretraining an object‑centric world model to extract stable pose priors, then integrating these priors into a pose‑aware low‑level policy. By automating data processing with foundation models, P2P‑T eliminates the need for human‑robot aligned data and achieves a 73% improvement over prior state‑of‑the‑art performance on complex real‑world tool manipulation tasks.
arXiv Computer Vision
Sep 4
RoboTok: An Internet-Scale Data Engine for Human Demonstration Retrieval and Dexterous Manipulation Learning
RoboTok is an internet‑scale data engine that retrieves human manipulation videos from the web to train dexterous robot policies. It learns a latent motion space from 3D hand trajectories in actor‑centered reference frames, allowing manipulation behaviors to be compared across different viewpoints, scenes, and occlusions while remaining compact for efficient search. Experiments show RoboTok retrieves more relevant demonstrations and improves downstream robot task success compared to existing retrieval methods.
arXiv AI
Jun 11
LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition
arXiv:2606. 11628v1 Announce Type: cross Abstract: The most widely-adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments.