HORIA presents a method for jointly reconstructing a human and an object in 4D from a single monocular video. It tackles two main challenges—complex, multi‑phase interaction semantics and occlusion‑induced incomplete views—by first segmenting the video into temporally grounded, motion‑aware interaction phases with text captions, then using these captions to guide an interaction‑aware re‑rendering that refines the reconstruction with novel‑view supervision from a text‑to‑video diffusion prior. The resulting framework delivers temporally coherent, accurate 4D reconstructions across diverse interaction scenarios, outperforming existing approaches.
arXiv:2606. 28215v1 Announce Type: cross Abstract: Extracting dynamic 4D object interactions from massive, in-the-wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs.
CARI4D is a category‑agnostic method that reconstructs spatially and temporally consistent 4D human‑object interactions from monocular RGB videos at metric scale. It introduces a pose hypothesis selection algorithm that integrates predictions from foundation models, refines them via a learned render‑and‑compare paradigm, and reasons about contacts to satisfy physical constraints. Experiments demonstrate a 38% improvement on in‑distribution data and 36% on unseen data compared to prior work, and the model generalizes to unseen categories, enabling zero‑shot application to internet videos.
arXiv:2609.15032v1 Announce Type: new Abstract: Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either r...
The paper introduces 4DStreamCtrl, a system that unifies camera motion, object trajectories, and depth into a single 3D point‑track representation, enabling joint control, depth editing, and motion transfer in a single forward pass. By mining in‑the‑wild video for 3D motion supervision and encoding it with a lightweight Geometric Motion Head, the authors train a causal streaming student that can generate arbitrarily long videos in just four denoising steps, achieving 20 FPS on a single high‑end GPU for 480p video. This approach outperforms prior camera‑only, 2D, and offline‑3D methods in motion‑control precision while maintaining temporal coherence over hundreds of frames, thereby enabling interactive 4D‑controllable streaming generation for the first time.
arXiv:2609.38615v1 Announce Type: cross Abstract: Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Ex...
arXiv:2607. 01202v1 Announce Type: cross Abstract: We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos.
arXiv:2610.02160v1 Announce Type: new Abstract: Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through i...
Dream4D is a new framework for generating spatiotemporally coherent 4D content. It uses a two‑stage pipeline: first, few‑shot learning predicts optimal camera trajectories from a single image; second, a pose‑conditioned diffusion process creates geometrically consistent multi‑view sequences that are converted into a persistent 4D representation. The method uniquely combines rich temporal priors from video diffusion models with geometric awareness from reconstruction models, achieving higher quality metrics such as mPSNR and mSSIM compared to existing approaches.
Precise 3D spatial orchestration in text-to-video generation remains a significant challenge, particularly for multi-object scenes where semantic layout and temporal dynamics are often entangled. While existing depth-conditioned models achieve good structural fidelity, they necessitate dense, frame-accurate guidance that is labor-intensive to author for dynamic events involving deformable objects.
Real-world spatial intelligence requires agents to understand scenes from continuous video streams, where objects move, persist, disappear, and reappear over time. While recent spatial foundation models have enabled generalizable feed-forward 3D reconstruction, most streaming methods remain geometry-centric and lack temporally consistent object-level understanding.
arXiv:2608.18734v2 Announce Type: replace Abstract: 4D understanding and reasoning is a fundamental capability for embodied AI agents operating in dynamic physical environments. However, existing vis...
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries. By integrating a frozen video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on benchmark tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.