arXiv Computer Vision

CJRR-Net: A Candidate-Joint Reprojection Ranking for Future-View Selection in Human Pose Estimation

The paper introduces CJRR-Net, a network designed to select the most informative camera view for human pose estimation in multi‑camera setups. It uses current‑view pose data, camera layout, and geometric features derived from reprojecting the estimated 3D pose and its uncertainty into each candidate view. In experiments on the MPI‑INF‑3DHP dataset, CJRR‑Net outperforms baseline methods, achieving a 2D mean per‑joint position error of 105.18 pixels and the lowest error across occlusion groups.

arXiv AI
3d ago

Multi-View 3D Reconstruction Using V-JEPA Features

The paper investigates supervised RGB‑only multi‑view 3D reconstruction by integrating video‑pretrained features with camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, reporting improvements in trajectory and intrinsics errors but mixed effects on depth accuracy and raw Chamfer distance. The results emphasize that representation adaptation, shape alignment, and raw reconstruction must be evaluated separately before attributing gains to predictive video pretraining.

By Anwar Shamim, Jeffrin Sam
arXiv Computer Vision
Oct 28

UT-HMR: Uncertainty-Guided Temporal Modeling for Robust 3D Human Mesh Recovery

UT‑HMR is a one‑stage monocular 3D human mesh recovery framework that introduces uncertainty‑guided temporal modeling to address ambiguities caused by weak observations such as distance, occlusion, and motion blur. It predicts a Gaussian distribution over SMPL pose with a low‑rank-plus‑diagonal covariance, converting this into joint‑wise precision weights for heteroscedastic supervision while keeping inference deterministic. Temporal early fusion incorporates neighboring‑frame evidence before pose decoding, enabling the model to resolve pose ambiguities and achieve state‑of‑the‑art results on VIP‑HockeyPose‑v2, WorldPose, and competitive performance on 3DPW using a lightweight ViT‑B backbone.

By Zhibo Wang, Sirisha Rambhatla, Yuhao Chen, Maria Koshkina, David Clausi
Hugging Face Trending Papers
6d ago

Multi-View 3D Reconstruction Using V-JEPA Features

The paper investigates supervised RGB‑only multi‑view 3D reconstruction by leveraging video‑pretrained features and a pipeline that includes camera estimation, scene aggregation, epipolar transport, and residual pointmaps. Experiments on CO3D and indoor datasets compare global versus spatial camera regression and a joint relative‑objective configuration, revealing trade‑offs in trajectory, depth, and Chamfer distance metrics. The results emphasize that improvements in reconstruction stem from specific representation adaptations and component interactions rather than solely from predictive video pretraining.

arXiv AI
Oct 31

SituHOI: Scalable Capture of Scene-Situated Full-Body Human-Object Interaction

SituHOI is a large‑scale RGB‑D dataset that captures full‑body human‑object interactions in 16 indoor rooms, comprising 18,429 annotated clips and about 86 hours of activity. The dataset pairs moving‑camera recordings with static scene scans, object geometry, language descriptions, and estimated human and object trajectories, all registered in a shared floor‑aligned frame via a pipeline that integrates camera localization, depth‑based alignment, human mesh recovery, and object tracking. The data enable the FlexHOI‑Bench benchmark for goal‑conditioned joint motion prediction, and a reference predictor, SharedHOI, demonstrates strong performance on world‑space accuracy and human‑object coordination. whyItMatters":"The dataset provides scalable, scene‑aligned supervision for modeling human‑object interaction, addressing a key limitation of existing datasets and enabling more accurate motion prediction benchmarks."

By Qifan Wang, Qihang Fang, Boao Shi, Yushi Du, Pei Zhou, Siyan Dong, Yanchao Yang
arXiv Computer Vision
Nov 6

Kineo: Calibration-Free Metric Motion Capture From Sparse RGB Cameras

Kineo is a fully automatic, calibration‑free pipeline that performs markerless motion capture using unsynchronized, uncalibrated consumer‑grade RGB cameras. It uses 2D keypoints from existing detectors to jointly calibrate cameras (including distortion), reconstruct 3D keypoints, and generate dense scene pointmaps at metric scale. The method introduces confidence‑aware subsampling, graph‑based optimization, and a pairwise reprojection consistency score to achieve robust calibration and accurate metric scale recovery, achieving significant error reductions on EgoHumans and Human3.6M datasets while remaining computationally efficient.

By Charles Javerliat, Pierre Raimbaud, Guillaume Lavou\'e
arXiv AI
Oct 31

Aligning Misaligned Benchmarks for Single-View Novel View Synthesis

The paper addresses the lack of standardized evaluation for single‑view novel view synthesis by highlighting the importance of aligning a method’s reference frame with the ground‑truth camera pose. It proposes using traditional 2D‑to‑2D and 2D‑to‑3D image matching to align 3D reconstructions and restricts evaluation to co‑visible regions where a correct ground truth exists. Experiments show that with proper alignment, 3D Gaussian‑based methods reach state‑of‑the‑art performance, and a simple baseline using a video generator and geometry estimator matches specialized novel‑view synthesis models without task‑specific training.

By Yang-Che Tseng, Erqun Dong, Jeong-gi Kwak, Gopal Sharma, Shakiba Kheradmand, Stephan Garbin, Hossam Isack, Kwang Moo Yi
arXiv Computer Vision
Aug 27

PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

PIVOT is a new multi‑trajectory dataset and evaluation framework that captures real‑world scenes with diverse camera paths, preserving both sensor‑derived measured poses and COLMAP‑optimized poses along with calibrated and optimized intrinsics. It defines three benchmark families—seen vs. unseen trajectory generalization, measured vs. optimized pose sensitivity, and calibrated vs. optimized intrinsics sensitivity—and introduces a directed pose‑space Chamfer distance to assess pose coverage. The first version of PIVOT includes five scenes recorded with a DJI Mini 4 Pro and offers an open processing and Nerfstudio‑based evaluation toolchain, revealing a consistent quality gap between held‑out and unseen trajectories and significant sensitivity to pose source and camera intrinsics.

By Mary Raymond
arXiv Computer Vision
Oct 16

Benchmarking Efficient & Effective Camera Pose Estimation Strategies for Novel View Synthesis

The paper introduces a benchmark for evaluating Structure-from-Motion (SfM) methods used in novel view synthesis (NVS) such as NeRFs and 3DGS. It compares classical SfM, which relies on local feature matches and bundle adjustment, with recent neural network approaches that regress camera parameters directly for faster but less accurate results. The study finds that reducing feature counts speeds up classical SfM without sacrificing pose accuracy, and that combining neural network initial estimates with classical refinement yields the best balance of efficiency and effectiveness.

By Jhacson Meza, Martin R. Oswald, Torsten Sattler
arXiv Computer Vision
Nov 9

MetricPose: Flexible 3D Human Pose Estimation with Depth and Geometric Cues

MetricPose is a 3D human pose estimator that predicts metrically accurate joint positions in the camera coordinate frame by combining learned RGB priors, direct depth measurements, and geometric constraints such as ground contact. The model extends a pretrained full-body estimator with optional metric depth and geometry-processing modules, enabling unified support for RGB, RGB‑Ground, and RGB‑D inputs while preserving RGB‑only operation. Compared to the SAM 3D Body foundation model, MetricPose reduces camera‑local joint‑position error by 12.16%, 31.00%, and 54.61% in RGB, RGB‑Ground, and RGB‑D modes respectively.

By Shifan Zhu, Sabad Modi, Hang Yang, Arushi Agrawal, Steven Gao, Meghan Huber, Nathan Wycoff, Donghyun Kim