arXiv Computer Vision September 4, 2026 By Howard Qian, Yiting Chen, Yunfei Xie, Kejia Ren, Podshara Chanrungmaneekul, Gaotian Wang, Bowen Wen, Chen Wei, Kaiyu Hang

RoboTok: An Internet-Scale Data Engine for Human Demonstration Retrieval and Dexterous Manipulation Learning

Read the original on arXiv Computer Vision →

RoboTok is an internet‑scale data engine that retrieves human manipulation videos from the web to train dexterous robot policies. It learns a latent motion space from 3D hand trajectories in actor‑centered reference frames, allowing manipulation behaviors to be compared across different viewpoints, scenes, and occlusions while remaining compact for efficient search. Experiments show RoboTok retrieves more relevant demonstrations and improves downstream robot task success compared to existing retrieval methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Machine Learning

Jun 10

Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations

arXiv:2606. 10614v1 Announce Type: cross Abstract: Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots.

By Beomjun Kim, Seong Hyeon Park, Seunghoon Sim, Seungjun Moon, Sanghyeok Lee, Jinwoo Shin

llms agents robotics fine-tuning benchmarks

arXiv AI

Jun 11

LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition

arXiv:2606. 11628v1 Announce Type: cross Abstract: The most widely-adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments.

By Harsh Gupta, Guanya Shi, Wenzhen Yuan

robotics

arXiv AI

Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo

robotics multimodal

arXiv AI

Aug 17

AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning

arXiv:2608. 14028v1 Announce Type: cross Abstract: Dexterous manipulation is a fundamental capability for embodied intelligence, but scaling it remains difficult because robot demonstrations are expensive to collect and action spaces vary across embodiments.

By Zhiyue Zhao, Jingyi Wu, Hairuo Liu, Mingyu Liu, Liyang Li, Hengdi Zhang, Tong He, Zhengxue Cheng

robotics multimodal safety

arXiv AI

Jun 8

What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?

arXiv:2606. 06627v1 Announce Type: cross Abstract: Human video datasets used for cotraining robot manipulation policies largely consist of curated demonstrations where motions are orchestrated to resemble robot behavior and 3D hand poses are captured with specialized hardware.

By Richard Li, Aditya Prakash, Andrew Wen, Saurabh Gupta, Yilun Du, Pulkit Agrawal

robotics

arXiv Machine Learning

Jul 31

CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations

arXiv:2505. 04999v2 Announce Type: replace-cross Abstract: Learning robot control policies from demonstrations typically requires action-labeled expert data, which is expensive to collect through teleoperation.

By Anthony Liang, Pavel Czempin, Matthew M. Hong, Yutai Zhou, Jingzhen Wang, Erdem Biyik, Stephen Tu

robotics

One email a morning, machine-written

Related stories

Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations

LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning

What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?

CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations