arXiv:2608.18734v2 Announce Type: replace
Abstract: 4D understanding and reasoning is a fundamental capability for embodied AI agents operating in dynamic physical environments. However, existing vis...
By Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe, Hasitha Gallella, Dulanga Weerakoon, Vigneshwaran Subbaraju, Ranga Rodrigo
Real-world spatial intelligence requires agents to understand scenes from continuous video streams, where objects move, persist, disappear, and reappear over time. While recent spatial foundation models have enabled generalizable feed-forward 3D reconstruction, most streaming methods remain geometry-centric and lack temporally consistent object-level understanding.
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes clean video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries for temporal flexibility. By incorporating a frozen pretrained video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on these tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.
Dream4D is a new framework for generating spatiotemporally coherent 4D content. It uses a two‑stage pipeline: first, few‑shot learning predicts optimal camera trajectories from a single image; second, a pose‑conditioned diffusion process creates geometrically consistent multi‑view sequences that are converted into a persistent 4D representation. The method uniquely combines rich temporal priors from video diffusion models with geometric awareness from reconstruction models, achieving higher quality metrics such as mPSNR and mSSIM compared to existing approaches.
By Xiaoyan Liu, Kangrui Li, Jiaxin Liu, Yuehao Song, Yujie Xing
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries. By integrating a frozen video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on benchmark tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.
By Yitong Xia, Zeren Jiang, Chuanxia Zheng
arXiv:2604.18484v2 Announce Type: replace
Abstract: Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from co...
By Kangan Qian, ChuChu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangjun Ye, Guang Chen, Long Chen, Diange Yang