AuroraWorld proposes a method for generating egocentric manipulation videos from large‑scale exocentric footage, addressing challenges such as viewpoint changes, occlusions, complex object interactions, and sparse observations. The approach aims to provide scalable, physically consistent data for visuomotor policy learning without the high costs of dedicated egocentric hardware. Code will be released publicly on GitHub.
arXiv:2606. 08057v1 Announce Type: cross Abstract: Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets.
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
arXiv:2610.27946v1 Announce Type: new Abstract: Egocentric action transfer aims to decouple the semantic action from egocentric observations and reproduce them in new visual contexts, enabling scalab...
arXiv:2607. 08436v1 Announce Type: cross Abstract: Egocentric human data offers scalable supervision for robot manipulation.
arXiv:2610.07217v1 Announce Type: cross Abstract: Despite its promise for scaling robot learning, egocentric manipulation data is still scarce today. Collection at scale requires vertically integrati...
Ego-Exo4D is a large-scale dataset that offers synchronized egocentric and multi-view exocentric video for applications such as skill learning, procedural activity understanding, and embodied AI. The original dataset only includes sparse 3D human pose annotations, making dense motion reconstruction challenging. To address this, the authors introduce Ego-Exo4D-HM, a new dataset containing 4D human motion reconstructions for the Ego-Exo4D captures, along with a reconstruction pipeline and accompanying code and documentation.
MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.
AWAM (Active World-Action Model) is a virtual active-perception framework that plans where a robot should observe and imagines the visual input along that trajectory before acting. It predicts task-related virtual camera motion, uses it as a structured query into a geometry-conditioned world model, and compresses the resulting spatial prior into fixed memory for later fusion with live observations via cross-attention. The model achieves competitive success rates on LIBERO-Occ, LIBERO-Plus, RLBench-OG Occlusion1, and real-robot trials, demonstrating improved robustness to occlusions and distribution shifts in less controlled environments.
ZimaBlue is a scalable framework that learns generalizable World Action Models (WAMs) from large-scale egocentric videos. It follows a three-stage curriculum: causal video pre‑training, video‑action mid‑training with a unified action representation, and final specialization to a target robot. The system employs an asynchronous Slow‑Fast architecture to enable real‑time 30 Hz action prediction, achieving a jump in real‑robot zero‑shot success from 36.1% to 77.8% when leveraging over 120,000 hours of embodied video.
arXiv:2606. 20118v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies have shown strong potential for general-purpose manipulation, yet they often fail on novel, out-of-distribution objects whose appearance or geometry deviates from the training distribution.
Ego2Act is a new benchmark that tests video generation models on goal‑directed, egocentric manipulation tasks. It contains 2,640 videos from 110 real‑world tasks, each requiring multiple steps of object manipulation to achieve a high‑level goal. The benchmark includes Ego2ActJudge, a reference‑free evaluation pipeline that better aligns with human judgments of task completion and physics plausibility.
arXiv:2610.18950v1 Announce Type: cross Abstract: Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting robot demonstrations at scale remains...