← Back to all news
arXiv Computer Vision By Zhangsihao Yang, Mengyi Shan

CoaG: Cylinders on a Grid: Coarse 3D Layout Control for Video Generation

Read the original on arXiv Computer Vision →

CoaG introduces a minimal geometric interface— a ground plane grid and a single cylinder per person—to control the placement and movement of people and the camera in generated videos. Users draw the grid, position cylinders, and animate them over 81 frames; the model then renders a photorealistic video that follows these layouts and camera paths, with appearance guided by a text prompt and a background image. To train the system, the authors automatically generate 2,000 captioned clips, reconstruct geometry from each clip, and fine‑tune a LoRA on Wan2.2‑Fun‑Control, enabling the model to respect user‑drawn layouts and camera motions on unseen clips.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Sep 25

CoaG: Cylinders on a Grid for Coarse 3D Layout Control in Video Generation

The paper introduces CoaG, a method that allows users to control the placement and movement of people and the camera in generated videos using minimal geometric input—a ground plane grid and one cylinder per person. By drawing a grid, positioning cylinders, and animating them over 81 frames, the model produces photorealistic videos that respect the specified layout and motion, while appearance is guided by a text prompt and a background image. To train the system, the authors automatically generate 2,000 caption–video pairs, extract geometry via tracking and reconstruction, and fine‑tune a LoRA on a text‑to‑video model to follow user‑drawn layouts and camera paths.

By Zhangsihao Yang, Mengyi Shan
agentsdiffusionfine-tuning
More like this →
arXiv AI
Oct 2

Generative Cinematographer: Composing Camera and Object Motion in 3D

Generative Cinematographer (GenCine) lifts a single image into an editable 3D scene scaffold, allowing artists to jointly author camera paths and foreground motion using local 3D motion handles. The system projects these controls into guidance maps that encode handle positions and colors across frames, enabling a pretrained video model to follow camera-relative, piecewise-rigid motion without physics simulation. Training recovers controls from real videos and synthetic ground-truth geometry, and experiments demonstrate consistent camera-relative motion, improved geometric consistency, and strong controllability across diverse real-world scenes.

By Jiahan Zhang, Chaohao Yang, Namitha Guruprasad, Vivekjyoti Banerjee, Trong-Tung Nguyen, Alan Yuille, Anand Bhattad
diffusionfine-tuning
More like this →
arXiv Computer Vision
Sep 30

MUGEN: Interactive Panoramic World Exploration via Camera Control

MUGEN is a large‑scale real‑world dataset of over 1,300 hours of 4K panoramic videos with rich semantic and geometric annotations, designed to support interactive 360° world exploration. Wan360 is a camera‑controllable panoramic video generation model built on MUGEN, featuring ERP‑aware components (periodic longitude RoPE, ERP‑aware padding, random roll yaw) and a panoramic Plücker embedding for camera motion. Together, they address gaps in data and model support for immersive, temporally coherent 360° video generation along user‑specified camera trajectories.

By Jiaming Tan, Zhen Li, Shuwei Shi, Minggui Teng, Siqi Yang, Yuwei Wu, Bo Zheng, Chuanhao Li, Kaipeng Zhang
ragdiffusion
More like this →