arXiv Computer Vision

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

The paper introduces a zero‑shot video restoration and enhancement framework that leverages a text‑to‑image latent diffusion model along with multi‑modal references. It employs dual prompt tuning inversion and sampling to cut inference time to about one‑third of the original, while also strengthening performance and temporal consistency. Additional techniques such as texture‑aware video token merging, referenced self‑attention, and referenced token merging further improve temporal coherence across frames.

arXiv AI
Sep 18

CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Models

CleanVideo introduces a selective erasure framework for text-to-video diffusion models, addressing the challenge of removing undesired visual concepts from videos. The method uses a low-dimensional subspace intervention guided by a tri-modal gating mechanism that jointly considers spatiotemporal visual features, timestep signals, and textual semantics to decide where, when, and whether to intervene. Experiments on three video diffusion models demonstrate that CleanVideo effectively erases target concepts while preserving visual fidelity, temporal coherence, and outperforming existing baselines in both frame-level and video-level evaluations, even under concept-recovery attacks.

By Junchi Liao, Hongji Li, Wenrui Zhou, Lijie Hu
arXiv AI
Sep 3

ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generation

ContextAnyone is a context‑aware diffusion framework that treats a reference image as an explicitly preserved appearance anchor rather than a simple conditioning signal. By jointly reconstructing the reference image and generating the target video within a shared diffusion transformer, it provides direct supervision for maintaining identity and fine‑grained appearance throughout denoising. The method introduces asymmetric information flow and Gap‑RoPE positional representations to keep the reference stable while allowing selective access by video tokens, and demonstrates improved identity and appearance consistency on an OpenVid‑HD benchmark.

By Ziyang Mai, Yu-Wing Tai
arXiv AI
Sep 25

TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution

TOLA is a diffusion‑based text image super‑resolution method that eliminates iterative image‑text diffusion by using a one‑step latent adaptation framework. It employs a confidence‑weighted text conditioning module to build a reliable semantic condition and a lightweight latent residual correction module to fix structured residual errors, thereby preserving text fidelity. Experiments show TOLA outperforms existing diffusion‑based TSR methods, achieving at least 2.72 dB higher PSNR on the CTR‑TSR‑Test benchmark.

By Yike Xu, Yue Shi, Yong Guo, Jiezhang Cao
arXiv Computer Vision
Sep 24

Zero-Shot Object Removal via Attention Masking, Latent Anchoring, and Refinement

This paper presents a zero‑shot framework for removing objects from real images using a frozen Stable Diffusion model, avoiding any task‑specific training. The pipeline combines SAM‑based mask construction, BLIP caption conditioning, DDIM inversion, background‑weighted masked null‑text optimization, decoder self‑attention masking, hard outside‑mask latent anchoring, and localized renoise‑denoise refinement. Experiments show effective removal of objects and context‑consistent replacement, with background‑weighted NTI especially helpful for complex backgrounds and repeated refinement reducing residual artifacts.

By Arman Taghizadeh (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Ulf Krumnack (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Kai-Uwe K\"uhnberger (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany)
arXiv Computer Vision
Sep 30

FastVR: Efficient Streaming Video Restoration with One-Step Diffusion

FastVR is a streaming video restoration framework that uses a one‑step diffusion model to achieve strong restoration quality and temporal consistency while processing 1080p video at 11 FPS on a single H20 GPU. It addresses efficiency bottlenecks by combining a lightweight VAE with chunk‑wise causal attention, and improves inference speed and restoration quality through velocity consistency regularization and continuous trajectory learning during training. Experiments demonstrate that FastVR outperforms diffusion baselines in efficiency and achieves state‑of‑the‑art performance on both synthetic and real‑world benchmarks.

By Xiaoxu Chen, Qin Yang, Haoran Bai, Sibin Deng, Ying Chen
arXiv Computer Vision
Sep 21

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Edit‑VAR is a training‑free, inversion‑free framework that uses a pretrained visual autoregressive video model for text‑guided video editing. It encodes the source video into multi‑scale discrete tokens and applies probability‑guided conditional token replacement, attention‑guided token‑wise and scale‑aware modulation, and scale‑decoupled generation to preserve source appearance while enabling precise edits. The method also includes residual‑guided token pruning to reduce inference cost, and experimental results show it outperforms existing training‑free video editing methods in fidelity, source preservation, temporal coherence, and efficiency.

By Chongbo Zhao, Jiangming Wang, Xilai Wang, Xinyu Wang, Jingyi Tang, Chunjie Hao, Pengjie Song, Yue Ma