EgoGen is a synthetic data generator designed for egocentric perception tasks, producing accurate ground‑truth training data by simulating natural human movements that steer virtual cameras. Its core is a human motion synthesis model that uses egocentric visual inputs to sense 3D environments, combined with collision‑avoiding motion primitives and a two‑stage reinforcement learning approach for closed‑loop perception and movement. The system eliminates the need for pre‑defined global paths, works in dynamic settings, and supports tasks such as mapping, localization, camera tracking, and human mesh recovery from first‑person views.
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
arXiv:2607. 17790v1 Announce Type: cross Abstract: Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environment.
Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environment. A holistic and efficient multimodal model capable of reconstructing this 4D representation is therefore highly desirable.
arXiv:2506.07886v4 Announce Type: replace Abstract: Understanding multimodal signals in egocentric vision, such as RGB video, depth, camera poses, and gaze, is essential for applications in augmented...
arXiv:2609.01276v1 Announce Type: new Abstract: Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Exis...
MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.
arXiv:2611.08477v1 Announce Type: new Abstract: Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts...
Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts pretrained video generators to unify action-cond...
ESTHER is an end‑to‑end model for egocentric stereo hand reconstruction that integrates stereo geometry, temporal reasoning, and a suitable output representation for wearable cameras. It is trained on pseudo‑labels from a calibrated pipeline and introduces the ESTHER3D benchmark, which pairs a large in‑the‑wild training set with a motion‑capture test set of true metric ground truth. Experiments demonstrate state‑of‑the‑art accuracy, strong generalization, and robustness to missing views, dropped frames, and extreme lighting or motion blur, while preserving metric scale even when reduced to a single monocular view.
arXiv:2610.27946v1 Announce Type: new Abstract: Egocentric action transfer aims to decouple the semantic action from egocentric observations and reproduce them in new visual contexts, enabling scalab...
arXiv:2606.08495v2 Announce Type: replace-cross Abstract: Humanoid robots require whole-body motions that adapt to scene context, task requirements, and user intent. Motion tracking reproduces specif...
Ego2Act is a new benchmark that tests video generation models on goal‑directed, egocentric manipulation tasks. It contains 2,640 videos from 110 real‑world tasks, each requiring multiple steps of object manipulation to achieve a high‑level goal. The benchmark includes Ego2ActJudge, a reference‑free evaluation pipeline that better aligns with human judgments of task completion and physics plausibility.