← Back to all news
arXiv Computer Vision By Chen Xu, Yunqi Li, Binbin Huang, Brent Yi, Shenghua Gao, Yi Ma

Video-Conditioned Generative Joint 2D-3D Hand Motion Recovery

Read the original on arXiv Computer Vision →

The paper introduces JoHan, a generative framework that directly recovers 2D and 3D hand motion from video sequences without intermediate per‑frame pose predictions. By jointly learning temporal dynamics and 2D‑3D correspondence, JoHan generates aligned pose sequences that improve temporal consistency and enable accurate estimation of the hand’s global position and orientation relative to the camera. Experiments on challenging benchmarks show that JoHan achieves higher accuracy and faster performance, producing smoother hand‑motion dynamics while maintaining high per‑frame pose accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Aug 28

Grasp in Gaussians: Fast Monocular Reconstruction of Dynamic Hand-Object Interactions

Grasp in Gaussians (GraG) is a fast, robust method for reconstructing dynamic 3D hand‑object interactions from a single monocular video. It leverages pretrained hand and object priors and represents the scene with a compact Sum‑of‑Gaussians (SoG) model, enabling efficient tracking while preserving geometric fidelity. Experiments show GraG achieves temporally coherent reconstructions on long sequences 4.4×–38.9× faster than prior work.

By Ayce Idil Aytekin, Xu Chen, Zhengyang Shen, Thabo Beeler, Helge Rhodin, Rishabh Dabral, Christian Theobalt
benchmarks
More like this →
arXiv Computer Vision
Sep 7

MINT: A Unified Model for World-Space Camera and Hand Motion Estimation from Scalable Egocentric Pipeline Supervision

MINT is a foundation model that directly predicts world-space two-hand trajectories from egocentric RGB video, jointly estimating camera motion, hand states, and hand presence in a single spatiotemporal representation. It uses an open-source labeling pipeline, EGOPIPELINE, to generate large-scale pseudo-labels for pretraining, followed by fine-tuning on a small set of high-quality joint annotations. The model outperforms existing multi-stage approaches in accuracy and speed, and generalizes zero‑shot to unseen egocentric datasets.

By Zijie Zhu, Weiren Cai, Yizhou Wang, Zhenjie Yang, Yide Liu, Jiahao Chen, Guanqi He
roboticsfine-tuningbenchmarks
More like this →