arXiv Computer Vision By Shengxiang Ji, Boyang Wang, Haiyang Xu, Bingnan Li, Yucheng Mao, Zeyuan Chen, Xiaojun Shan, Xiang Zhang, Gang Hua, Jianwen Xie, Zezhou Cheng, Zhuowen Tu

LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation

Read the original on arXiv Computer Vision →

LIFT is a unified image‑to‑video generation framework that adds Layout‑In‑Future control, letting users specify what should appear and where in a future view. It addresses the limitation of existing camera controls and text prompts by using the last‑frame layout as an explicit signal for the desired future scene, especially under large viewpoint changes. To handle sparse layout guidance, LIFT employs on‑policy self‑distillation to transfer knowledge from a dense‑layout teacher to a last‑frame‑layout student, and introduces the LIFT‑Vista dataset with large viewpoint changes and consistent layout annotations. Experiments demonstrate that LIFT improves video quality, future‑layout controllability, and camera controllability compared to other methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Sep 2

CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling

CameraEditor is a new framework that transforms camera-controlled image editing into a temporal sequence prediction problem. By using video diffusion models, it incorporates a geometric perception module and dynamic reference routing to create precise visual references through dynamic panorama cropping. The method also inserts intermediate transition frames to handle large perspective shifts, maintaining content identity and spatial coherence, and is evaluated on a dataset of 5,760 instances with a benchmark of 462 test cases, achieving state‑of‑the‑art performance.

By Xin Shen, Chengyou Jia, Keshuo Xing, Zifeng Zhu, Changliang Xia, Bowen Ping, Zhuohang Dang, Hangwei Qian, Minnan Luo
Hugging Face Trending Papers
Sep 10

World in World: Explore the World with World Models

World in World introduces a training‑free inference interface that lets users control autoregressive video world models from new viewpoints. By converting diverse control signals—source‑video observations, target‑view projections, geometry renderings, and retrieved states—into camera‑ and time‑labelled visual tokens, the system uses a frozen causal video model’s self‑attention to maintain synchronization, complete unseen regions, and recover past appearances. The method supports camera‑controlled rerendering, long‑horizon revisiting, and human‑motion transfer while preserving perceptual quality, temporal consistency, and camera‑following accuracy.

Hugging Face Trending Papers
Jul 27

CameraAnything: Refilming Videos with Arbitrary Camera Control

We introduce CameraAnything, the first unified framework for camera controlled video editing that enables joint control of both intrinsic and extrinsic camera parameters. Existing approaches either rely on expensive 3D reconstruction to achieve full camera functionality or restrict editing to extrinsic parameter manipulation.

Hugging Face Trending Papers
Jun 17

LooseControlVideo: Directorial Video Control using Spatial Blocking

Precise 3D spatial orchestration in text-to-video generation remains a significant challenge, particularly for multi-object scenes where semantic layout and temporal dynamics are often entangled. While existing depth-conditioned models achieve good structural fidelity, they necessitate dense, frame-accurate guidance that is labor-intensive to author for dynamic events involving deformable objects.