arXiv AI By Gen Li, Kaifeng Zhao, Siwei Zhang, Xiaozhong Lyu, Mihai Dusmanu, Yan Zhang, Marc Pollefeys, Siyu Tang

EgoGen: An Egocentric Synthetic Data Generator

Read the original on arXiv AI →

EgoGen is a synthetic data generator designed for egocentric perception tasks, producing accurate ground‑truth training data by simulating natural human movements that steer virtual cameras. Its core is a human motion synthesis model that uses egocentric visual inputs to sense 3D environments, combined with collision‑avoiding motion primitives and a two‑stage reinforcement learning approach for closed‑loop perception and movement. The system eliminates the need for pre‑defined global paths, works in dynamic settings, and supports tasks such as mapping, localization, camera tracking, and human mesh recovery from first‑person views.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jul 2

EgoSim: Egocentric World Simulator for Embodied Interaction Generation

arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.

By Jinkun Hao, Mingda Jia, Ruiyan Wang, Hongrui Zhu, Jiafei Cao, Xihui Liu, Ran Yi, Lizhuang Ma, Jiangmiao Pang, Xudong Xu
arXiv Computer Vision
Sep 7

MINT: A Unified Model for World-Space Camera and Hand Motion Estimation from Scalable Egocentric Pipeline Supervision

MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.

By Zijie Zhu, Weiren Cai, Yizhou Wang, Zhenjie Yang, Yide Liu, Jiahao Chen, Guanqi He