EgoKit is a toolkit that standardizes egocentric video recording across six different host devices, providing a uniform recording workflow and log format. It supports synchronized ego-view and wrist-view capture, and on XR headsets it logs head pose and hand tracking aligned to the video. The toolkit includes accessories such as wrist cameras, a head strap, and a USB-C hub, enabling wrist-view capture without custom hardware, and has been validated to train robot policies comparable to those trained on other datasets.
arXiv:2610.27622v1 Announce Type: cross Abstract: Handheld demonstration devices decouple data collection from the robot, but conventional designs burden the hand with a wrist camera, cables and a po...
MEgoVista is an offline pipeline that converts a single unprepared egocentric video into metric two‑hand and head motion within a gravity‑aligned world frame. It uniquely reconstructs motion in environments beyond studio volumes, uses calibrated stereo for absolute scale, and evaluates its outputs against independent optical capture to audit accuracy. The system thus expands the settings where high‑fidelity hand‑motion labels can be generated from natural, head‑worn recordings.
arXiv:2610.07217v1 Announce Type: cross Abstract: Despite its promise for scaling robot learning, egocentric manipulation data is still scarce today. Collection at scale requires vertically integrati...
arXiv:2607. 15868v1 Announce Type: cross Abstract: Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR.
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
arXiv:2608. 15614v1 Announce Type: cross Abstract: The use of multimodal LLMs (MLLMs) for egocentric video understanding with wearable devices is constrained by the token budget.
arXiv:2609.39378v1 Announce Type: new Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolvin...
arXiv:2607. 08436v1 Announce Type: cross Abstract: Egocentric human data offers scalable supervision for robot manipulation.
WARP-VLA is a Vision‑Language‑Action model designed to be robust to variations in wrist‑camera viewpoints. It uses a Mixture‑of‑Experts architecture where each expert learns a view‑specific feature transformation, and a router selects the appropriate expert based on implicit view cues, eliminating the need for explicit camera extrinsic parameters. Experiments on the LIBERO benchmark show that WARP‑VLA doubles the success rate of pi‑0.5 from 39.2% to 78.3% under wrist‑view perturbations, and real‑robot tests confirm that the simulation‑learned adaptation transfers to diverse deployment settings.
INSPECT is a system that learns how a robot should choose its camera view during assembly inspection by observing a smart‑glasses assistant that answers part queries and guides the user. It uses techniques such as Presence‑Invariant TwinSwap for object evidence calibration, claim‑indexed supervision to separate evidence needs from camera changes, and object‑centered calibration to adapt view preferences to robot poses. In experiments on gearbox assemblies and angle‑grinder recordings, INSPECT outperforms other non‑oracle policies, improving view utility and decision accuracy.
HumanEgo is a framework that enables zero‑shot robot learning from short egocentric human videos by converting each demonstration into an entity‑level hand‑object interaction representation and training a flow‑matching policy with dense auxiliary objectives. The method is robot‑data‑free, hardware‑agnostic, and data‑efficient, achieving 92.5 % success on four real‑world tasks with only 30 minutes of human video per task and outperforming matched‑time robot teleoperation by 41 %. HumanEgo also robustly transfers zero‑shot across new robots, cameras, and environments, and is released as an open‑source tool for learning robot policies directly from human data.
AuroraWorld proposes a method for generating egocentric manipulation videos from large‑scale exocentric footage, addressing challenges such as viewpoint changes, occlusions, complex object interactions, and sparse observations. The approach aims to provide scalable, physically consistent data for visuomotor policy learning without the high costs of dedicated egocentric hardware. Code will be released publicly on GitHub.