arXiv Computer Vision
Aug 27

4DStreamCtrl: Interactive Video Generation with Online 4D Control

The paper introduces 4DStreamCtrl, a system that unifies camera motion, object trajectories, and depth into a single 3D point‑track representation, enabling joint control, depth editing, and motion transfer in a single forward pass. By mining in‑the‑wild video for 3D motion supervision and encoding it with a lightweight Geometric Motion Head, the authors train a causal streaming student that can generate arbitrarily long videos in just four denoising steps, achieving 20 FPS on a single high‑end GPU for 480p video. This approach outperforms prior camera‑only, 2D, and offline‑3D methods in motion‑control precision while maintaining temporal coherence over hundreds of frames, thereby enabling interactive 4D‑controllable streaming generation for the first time.

By Shiqian Li, Chenguo Lin, Zhiguang Liu, Yu Tang, Jiarong Ou, Rui Chen, Yixin Zhu
Hugging Face Trending Papers
Jul 21

IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer

Real-world spatial intelligence requires agents to understand scenes from continuous video streams, where objects move, persist, disappear, and reappear over time. While recent spatial foundation models have enabled generalizable feed-forward 3D reconstruction, most streaming methods remain geometry-centric and lack temporally consistent object-level understanding.

arXiv AI
Oct 2

Generative Cinematographer: Composing Camera and Object Motion in 3D

Generative Cinematographer (GenCine) lifts a single image into an editable 3D scene scaffold, allowing artists to jointly author camera paths and foreground motion using local 3D motion handles. The system projects these controls into guidance maps that encode handle positions and colors across frames, enabling a pretrained video model to follow camera-relative, piecewise-rigid motion without physics simulation. Training recovers controls from real videos and synthetic ground-truth geometry, and experiments demonstrate consistent camera-relative motion, improved geometric consistency, and strong controllability across diverse real-world scenes.

By Jiahan Zhang, Chaohao Yang, Namitha Guruprasad, Vivekjyoti Banerjee, Trong-Tung Nguyen, Alan Yuille, Anand Bhattad
arXiv AI
Oct 27

OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images

OmniShowcase introduces a method for generating omnidirectional videos of a specific subject from a handful of multi‑view images. It combines an automated data‑synthesis pipeline that creates training videos with diverse camera trajectories driven by the subject’s geometry, and a geometry‑conditioning technique that supplies the model with surface‑normal maps reconstructed from the reference views. The authors release Showcase‑4K, a large corpus of 3D‑asset‑based videos with 12 camera motions and 88 trajectory configurations, and ShowcaseBench, a held‑out evaluation set with source meshes and camera parameters.

By Junyeong Ahn, Geonwoo Kim, Kinam Kim, Gihwi Kim, Hoonjin Jung, Hyojin Jang, Jaegul Choo
arXiv Computer Vision
Sep 11

Dream4D: Lifting Camera-Controlled I2V towards Spatiotemporally Consistent 4D Generation

Dream4D is a new framework for generating spatiotemporally coherent 4D content. It uses a two‑stage pipeline: first, few‑shot learning predicts optimal camera trajectories from a single image; second, a pose‑conditioned diffusion process creates geometrically consistent multi‑view sequences that are converted into a persistent 4D representation. The method uniquely combines rich temporal priors from video diffusion models with geometric awareness from reconstruction models, achieving higher quality metrics such as mPSNR and mSSIM compared to existing approaches.

By Xiaoyan Liu, Kangrui Li, Jiaxin Liu, Yuehao Song, Yujie Xing