← Back to all news
arXiv Computer Vision By Jay Mahajan, Chang Liu, Rauf Makharov, Viraj Shah, Alexander Schwing, Svetlana Lazebnik

ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features

Read the original on arXiv Computer Vision →

ReaDiT Guidance is a lightweight framework that controls Diffusion Transformer (DiT) models by leveraging internal feature representations from a single DiT block. It steers image and video generation toward spatial targets such as depth, pose, or edge maps, and extends naturally to video generation for camera and motion control. Experiments show competitive or improved results compared to existing feature‑based and adapter‑based methods while using fewer parameters.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 16

Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models

The paper introduces an adaptive step schedule controller for text‑to‑image diffusion models, allowing the number of denoising steps to vary based on the complexity of the input prompt. By mixing step schedules of different sizes and monitoring error discrepancies at each timestep, the method switches schedules to maintain image quality while reducing inference time. Experiments on COCO and DiffusionDB demonstrate that this approach achieves faster generation without sacrificing visual fidelity.

By Kuluhan Binici, Cihan Acar, Shivam Aggarwal, Siying Liu, Tulika Mitra
diffusion
More like this →
arXiv Computer Vision
Aug 25

BulletTime: Decoupled Control of Time and Camera Pose for Video Generation

BulletTime presents a 4D‑controllable video diffusion framework that separates scene dynamics from camera pose, allowing precise manipulation of both temporal and spatial aspects of generated videos. The model conditions on continuous world‑time sequences and camera trajectories, integrating them via a 4D positional encoding in the attention layer and adaptive normalizations for feature modulation. A specially curated dataset with independently parameterized temporal and camera variations is used for training, and the resulting system demonstrates robust real‑world 4D control while maintaining high generation quality and surpassing prior methods in controllability.

By Yiming Wang, Qihang Zhang, Shengqu Cai, Tong Wu, Jan Ackermann, Zhengfei Kuang, Yang Zheng, Frano Raji\v{c}, Siyu Tang, Gordon Wetzstein
diffusionrobotics
More like this →