arXiv Computer Vision

Estimating Accurate Hand Pose in Camera Space with Vision Transformer

arXiv Computer Vision
Oct 9

VersaCamVLA: Camera-Configurable VLA Policies for Robotic Manipulation

VersaCamVLA introduces a camera‑configurable framework that separates camera‑set representation from action learning in Vision‑Language‑Action models. It learns a unified scene‑token interface that maps any number of posed RGB views into fixed‑size latent tokens using multi‑signal target‑view prediction and Wrist‑Augmented Pose Sampling. At deployment, a lightweight spatial encoder injects these tokens into a pretrained VLA, enabling robust manipulation without explicit 3D sensing or novel‑view rendering, and consistently outperforms prior methods across varying camera counts and unseen poses.

By Boyao Han, Chen Shi, Jingjing Qian, ZhuoTan Tian, Li Jiang
arXiv Machine Learning
Oct 23

EgoHand: Monocular 3D Hand Reconstruction from Egocentric Video

EgoHand is a monocular 3D hand reconstruction model that sets new state‑of‑the‑art results on egocentric video. It is trained on a curated 14‑million‑image corpus that blends synthetic data, in‑the‑wild video, and multi‑camera motion capture to balance diversity and supervision quality. The model uses SOMA‑Hand as its parametric representation, processes 2D hand crops with a transformer encoder, and incorporates camera cues and an auxiliary 2D branch to enhance geometric consistency and image alignment, achieving low inference latency and strong out‑of‑domain generalization on benchmarks such as HOT3D, ARCTIC, and H2O.

By Chao-Yeh Chen, Tiffany Chen, Lin Duan, Tianyi Hu, Yu-Hsiang Huang, Umar Iqbal, Jiefeng Li, Edy Lim, Yanchen Liu, Yihan Pang, Hang Qiu, Jun Saito, Shao Su, Qingsi Wang, Xiufeng Xie
arXiv Computer Vision
Oct 8

Video-Conditioned Generative Joint 2D-3D Hand Motion Recovery

The paper introduces JoHan, a generative framework that directly recovers 2D and 3D hand motion from video sequences without intermediate per‑frame pose predictions. By jointly learning temporal dynamics and 2D‑3D correspondence, JoHan generates aligned pose sequences that improve temporal consistency and enable accurate estimation of the hand’s global position and orientation relative to the camera. Experiments on challenging benchmarks show that JoHan achieves higher accuracy and faster performance, producing smoother hand‑motion dynamics while maintaining high per‑frame pose accuracy.

By Chen Xu, Yunqi Li, Binbin Huang, Brent Yi, Shenghua Gao, Yi Ma
arXiv Computer Vision
Sep 16

EventEgoHands++: Event-based Egocentric 3D Hand Mesh Reconstruction with Real Dataset

EventEgoHands++ is a new framework for reconstructing 3D hand meshes from egocentric event-based cameras. It introduces a Hand Detector that provides instance-level bounding boxes and masks for left and right hands, and an Adaptive Attention module that uses these detections to model spatial relationships and interactions. The authors extend the synthetic N-HOT3D dataset and create EEH‑R, a large real-world event-based egocentric hand dataset with about 1 million annotated frames, and show that their method outperforms existing baselines on both synthetic and real data.

By Ryosei Hara, Wataru Ikeda, Masashi Hatano, Mariko Isogawa
arXiv Computer Vision
Nov 2

EgoHSI3R: Online 4D Hand-Scene Interaction Reconstruction from Egocentric Video

EgoHSI3R is a unified feed‑forward framework that reconstructs 4D hand‑scene interactions from egocentric monocular video. It jointly estimates camera motion, dense scene geometry, world‑space bilateral MANO hands, and per‑vertex hand contact by maintaining a recurrent scene representation and using hand‑centric prompts and MANO vertex queries. The method achieves the lowest world‑space hand joint error on ARCTIC while offering faster inference than existing pipelines.

By Mingzhi Pei, Zhangchen Ye, Ruoqu Chen, Liu Cao, Mengdi Xu