arXiv Computer Vision

Taming Noise Initialization for Zero-shot Video Restoration with Image Diffusion Priors

The paper introduces a method for zero‑shot video restoration that improves upon prior approaches by using DDIM inversion on corrupted video frames to generate noise that preserves temporal correlations. An adaptive noise blending strategy is added to reduce degradation patterns while gradually moving toward Gaussian noise, and a measurement‑guided trajectory attention mechanism further enhances temporal consistency. Experiments on DAVIS, REDS4, and Set8 datasets show that this approach outperforms existing zero‑shot video restoration methods.

arXiv Computer Vision
Aug 28

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

The paper introduces a zero‑shot video restoration and enhancement framework that leverages a text‑to‑image latent diffusion model along with multi‑modal references. It employs dual prompt tuning inversion and sampling to cut inference time to about one‑third of the original, while also strengthening performance and temporal consistency. Additional techniques such as texture‑aware video token merging, referenced self‑attention, and referenced token merging further improve temporal coherence across frames.

By Cong Cao, Huanjing Yue, Xin Liu, Jingyu Yang
arXiv Computer Vision
Sep 30

FastVR: Efficient Streaming Video Restoration with One-Step Diffusion

FastVR is a streaming video restoration framework that uses a one‑step diffusion model to achieve strong restoration quality and temporal consistency while processing 1080p video at 11 FPS on a single H20 GPU. It addresses efficiency bottlenecks by combining a lightweight VAE with chunk‑wise causal attention, and improves inference speed and restoration quality through velocity consistency regularization and continuous trajectory learning during training. Experiments demonstrate that FastVR outperforms diffusion baselines in efficiency and achieves state‑of‑the‑art performance on both synthetic and real‑world benchmarks.

By Xiaoxu Chen, Qin Yang, Haoran Bai, Sibin Deng, Ying Chen
arXiv Computer Vision
Aug 25

Pixel-Space Diffusion via Observation Operators

Pixel‑Space Diffusion via Observation Operators introduces a new framework for pixel‑space diffusion models that addresses a scale‑time mismatch in existing methods. By replacing fixed full‑image supervision with a time‑indexed observation trajectory that progresses from coarse structures to the full image, the model aligns supervision with the natural recovery order of image details. The approach employs Gaussian‑Lanczos operators and a GL‑CoDA decoder to refine features progressively, resulting in faster convergence and higher generation quality, achieving an FID of 1.52 on ImageNet‑256.

By Shaojie Guo, Lichen Ma, Haoyang Tong, Yu He, Zipeng Guo, Xiaoan Liu, Feng Yan, Yu Guo, Fei Wang, Junshi Huang, Yan Wang
arXiv AI
Jul 24

RealVDeblur: One-Step Diffusion for Generalizable Real-World Video Deblurring

arXiv:2607. 20628v1 Announce Type: cross Abstract: Real-world video deblurring remains challenging due to diverse motion patterns, complex degradations, and the scarcity of realistic training data, yet robust restoration is critical for downstream pipelines such as mobile imaging and 3D reconstruction.

By Renbiao Jin, Mingxin Yang, Yutian Chen, Junhao Zhuang, Xin Cai, Mulin Yu, Linning Xu, Wenxian Yu, Danping Zou, Shi Guo, Tianfan Xue
arXiv AI
Aug 18

Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning

arXiv:2608. 14706v1 Announce Type: cross Abstract: Standard autoregressive video generation algorithms based on Diffusion and Flow Matching rely on rigid training objectives and static sampling schedules, limiting inference procedures from adapting to the data.

By Hansen Jin Lillemark, Alex Rojas, Zachary Novack, Runqian Wang, Yilun Du, Yian Ma, Taylor Berg-Kirkpatrick, Rose Yu
arXiv Computer Vision
Aug 25

GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors

GaussVid introduces a 3D-aware video restoration framework that enhances sparse-view 3D Gaussian Splatting (3DGS) reconstructions. By creating a large-scale 3DGS video dataset and employing a camera-conditioned geometric prior anchored on the first and last frames, the method injects spatial structure into video generation, ensuring geometrically grounded restoration across viewpoints. Experiments demonstrate superior pixel- and structure-level fidelity (PSNR/SSIM) and improved multi-view consistency compared to other video-prior restoration methods, while maintaining competitive perceptual quality (LPIPS).

By Xinhui Liu, Can Wang, Wei Jiang, Wei Wang, Dong Xu