Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos.
Real-world spatial intelligence requires agents to understand scenes from continuous video streams, where objects move, persist, disappear, and reappear over time. While recent spatial foundation models have enabled generalizable feed-forward 3D reconstruction, most streaming methods remain geometry-centric and lack temporally consistent object-level understanding.
TAPVid-MV is a new benchmark for tracking any point in 3D across multiple synchronized camera views. It comprises 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks, covering indoor and outdoor domains and derived from various modalities such as depth, LiDAR, SLAM, and simulation. The dataset is visually verified, and evaluation shows that current multi‑view trackers do not consistently outperform monocular trackers, highlighting geometry recovery as a key bottleneck.
By Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco, Jeet Thakwani, Rishabh Kabra, Andrew Zisserman, Joao Carreira, Siyu Tang, Carl Doersch, Gabriel Brostow
arXiv:2603.12064v3 Announce Type: replace
Abstract: We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple freely moving cameras -- a settin...
By Shuo Sun, Unal Artan, Malcolm Mielle, Achim J. Lilienthaland, Martin Magnusson
PURE is an unlabeled post‑training refinement framework that enhances feed‑forward 3D reconstruction models by improving depth and camera estimation without ground‑truth supervision. It distills structural knowledge from state‑of‑the‑art 2D priors into 4D architectures, using relative depth constraints, affine‑invariant geometry distillation, and pose refinement via feature correspondences. Experiments on synthetic and real‑world dynamic benchmarks show consistent improvements across multiple state‑of‑the‑art models, effectively mitigating domain shifts and unlocking their architectural limits in challenging in‑the‑wild scenes.
By Yaolong Zhu, Peng Li, Jiahao Lu, Chengfeng Zhao, Mengfei Li, Chao Wang, Wenyue Chen, Zekai Gu, Chengwei Ren, Ronggang Wang, Yuan Liu
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes clean video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries for temporal flexibility. By incorporating a frozen pretrained video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on these tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.
arXiv:2610.27594v1 Announce Type: new
Abstract: While feed-forward models excel in static 3D geometry estimation, extending them to dynamic 4D scenes requires modeling visual geometry as a causally e...
By Qiaowei Miao, Yijing Guo, Lipei Zhang, Qifeng Cheng, Yuru Xiao, Ziqian Ni, Sheng Yang
arXiv:2610.02160v1 Announce Type: new
Abstract: Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through i...
By Wei Cao, Hao Zhang, Vikram Voleti, Yuqun Wu, Mallikarjun B R, Shimon Vainer, Mark Boss, Yaoyao Liu
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries. By integrating a frozen video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on benchmark tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.
By Yitong Xia, Zeren Jiang, Chuanxia Zheng
Accurate 4D reconstruction remains challenging: jointly estimating camera motion, scene geometry, and non-rigid deformation entails highly nonconvex optimization, making global optimality difficult to...
Kineo is a fully automatic, calibration‑free pipeline that performs markerless motion capture using unsynchronized, uncalibrated consumer‑grade RGB cameras. It uses 2D keypoints from existing detectors to jointly calibrate cameras (including distortion), reconstruct 3D keypoints, and generate dense scene pointmaps at metric scale. The method introduces confidence‑aware subsampling, graph‑based optimization, and a pairwise reprojection consistency score to achieve robust calibration and accurate metric scale recovery, achieving significant error reductions on EgoHumans and Human3.6M datasets while remaining computationally efficient.
By Charles Javerliat, Pierre Raimbaud, Guillaume Lavou\'e
Forge4D is a feed‑forward model that reconstructs temporally aligned 4D human representations from uncalibrated sparse‑view videos, enabling both novel view and novel time synthesis. It achieves this by jointly streaming 3D Gaussian reconstruction with dense motion prediction, using learnable state tokens for temporal consistency and a self‑supervised retargeting loss for motion prediction. Extensive experiments confirm its effectiveness on in‑domain and out‑of‑domain datasets.
By Yingdong Hu, Yisheng He, Jinnan Chen, Weihao Yuan, Kejie Qiu, Zehong Lin, Siyu Zhu, Zilong Dong, Steven Hoi, Jun Zhang