EgoHSI3R is a unified feed‑forward framework that reconstructs 4D hand‑scene interactions from egocentric monocular video. It jointly estimates camera motion, dense scene geometry, world‑space bilateral MANO hands, and per‑vertex hand contact by maintaining a recurrent scene representation and using hand‑centric prompts and MANO vertex queries. The method achieves the lowest world‑space hand joint error on ARCTIC while offering faster inference than existing pipelines.
By Mingzhi Pei, Zhangchen Ye, Ruoqu Chen, Liu Cao, Mengdi Xu
arXiv:2606.19156v2 Announce Type: replace
Abstract: Dynamic 3D hand reconstruction from egocentric videos is essential for next-generation computing platforms such as AR/VR and AI glasses. Despite it...
By Jeongmin Bae, Seoha Kim, Marc Pollefeys, Mahdi Rad, Youngjung Uh, Taein Kwon
MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.
By Zijie Zhu, Weiren Cai, Yizhou Wang, Zhenjie Yang, Yide Liu, Jiahao Chen, Guanqi He
ESTHER is an end‑to‑end model for egocentric stereo hand reconstruction that integrates stereo geometry, temporal reasoning, and a suitable output representation for wearable cameras. It is trained on pseudo‑labels from a calibrated pipeline and introduces the ESTHER3D benchmark, which pairs a large in‑the‑wild training set with a motion‑capture test set of true metric ground truth. Experiments demonstrate state‑of‑the‑art accuracy, strong generalization, and robustness to missing views, dropped frames, and extreme lighting or motion blur, while preserving metric scale even when reduced to a single monocular view.
By Hongyu Ma, Hairong Qu, Shiqi Zhao, Yongsong Yang, Peng Yin
EgoHand is a monocular 3D hand reconstruction model that sets new state‑of‑the‑art results on egocentric video. It is trained on a curated 14‑million‑image corpus that blends synthetic data, in‑the‑wild video, and multi‑camera motion capture to balance diversity and supervision quality. The model uses SOMA‑Hand as its parametric representation, processes 2D hand crops with a transformer encoder, and incorporates camera cues and an auxiliary 2D branch to enhance geometric consistency and image alignment, achieving low inference latency and strong out‑of‑domain generalization on benchmarks such as HOT3D, ARCTIC, and H2O.
By Chao-Yeh Chen, Tiffany Chen, Lin Duan, Tianyi Hu, Yu-Hsiang Huang, Umar Iqbal, Jiefeng Li, Edy Lim, Yanchen Liu, Yihan Pang, Hang Qiu, Jun Saito, Shao Su, Qingsi Wang, Xiufeng Xie
arXiv:2605.18553v2 Announce Type: replace-cross
Abstract: Recovering world space 4D motion of two interacting hands from egocentric video is a fundamental capability for supervising robot policy lear...
By Huajian Zeng, Chaohua Yao, Yuantai Zhang, Jiaqi Yang, Rolandos Alexandros Potamias, Xingxing Zuo