arXiv AI

Wavelet Phase Diffusion for Structurally and Semantically Consistent Sim-to-Real Translation

Wavelet Phase Diffusion (ψ‑PD) is a new simulation‑to‑real translation method that operates in the Dual‑Tree Complex Wavelet Packet Transform domain, allowing spatially adaptive phase injection without global spectral interference. It also introduces Low‑Frequency Randomization to decouple the model from synthetic illumination priors, enabling realistic, in‑distribution real‑world appearance while preserving structural and semantic consistency. Trained on unpaired open‑domain data, ψ‑PD achieves superior realism and semantic consistency on vKITTI→KITTI image translation and improves CARLA video translation performance, reducing planner errors by over 5%.

arXiv Computer Vision
Aug 24

Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving

The paper introduces Driving with DINO (DwD), a framework that uses Vision Foundation Module (VFM) features to bridge simulation and real-world domains for autonomous driving video generation. It addresses the consistency‑realism dilemma by projecting VFM features onto a principal subspace, dropping high‑frequency texture elements, and applying a Random Channel Tail Drop to preserve structural detail. Additional components— a learnable Spatial Alignment Module and a Causal Temporal Aggregator— enhance control precision, spatial alignment, and temporal stability, reducing motion blur and ensuring realistic, consistent outputs.

By Xuyang Chen, Conglang Zhang, Chuanheng Fu, Zihao Yang, Kaixuan Zhou, Yizhi Zhang, Yanfeng Zhang, Mingwei Sun, Zhen Dong, Xiaoxiao Long, Zengmao Wang, Liqiu Meng
arXiv Computer Vision
Aug 21

Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal

arXiv:2608. 20212v1 Announce Type: new Abstract: High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections.

By Radim Spetlik, David Futschik, Radek Danecek, Feitong Tan, Ziqian Bai, Rohit Pandey, Yinda Zhang
arXiv Machine Learning
Aug 27

WAVE: Reversing the Guidance Hierarchy for Coarse-to-Fine Guided Depth Super-Resolution

WAVE introduces a multi-level discrete wavelet transform (ML‑DWT) to reverse the typical fine‑to‑coarse bias in guided depth super‑resolution. By consuming wavelet sub‑bands and semantic tokens in reverse order, it separates structure and detail reconstruction, applies semantic gating to high‑frequency bands, and fuses modalities via an invertible coupling mechanism. Experiments on multiple benchmarks show that WAVE matches or outperforms existing methods, especially at high upsampling factors where low‑resolution depth has minimal structure.

By Tayyab Nasir, Daochang Liu, Ajmal Mian
arXiv AI
Jul 24

RealVDeblur: One-Step Diffusion for Generalizable Real-World Video Deblurring

arXiv:2607. 20628v1 Announce Type: cross Abstract: Real-world video deblurring remains challenging due to diverse motion patterns, complex degradations, and the scarcity of realistic training data, yet robust restoration is critical for downstream pipelines such as mobile imaging and 3D reconstruction.

By Renbiao Jin, Mingxin Yang, Yutian Chen, Junhao Zhuang, Xin Cai, Mulin Yu, Linning Xu, Wenxian Yu, Danping Zou, Shi Guo, Tianfan Xue
arXiv AI
Oct 9

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

VINCIE‑NExT is a unified framework that transfers image editing capabilities to video editing by using in‑context visual demonstrations. It decomposes video editing into a chain of sub‑tasks (Video → Image → Image → Video), routing editing intent through the image domain and enabling joint training from heterogeneous image and video data under a single diffusion objective. The method introduces a novel position encoding to link image demonstrations and video frames, allowing pixel‑faithful appearance propagation, and supports test‑time scaling through progressive diffusion stages.

By Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng
arXiv AI
Sep 3

ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generation

ContextAnyone is a context‑aware diffusion framework that treats a reference image as an explicitly preserved appearance anchor rather than a simple conditioning signal. By jointly reconstructing the reference image and generating the target video within a shared diffusion transformer, it provides direct supervision for maintaining identity and fine‑grained appearance throughout denoising. The method introduces asymmetric information flow and Gap‑RoPE positional representations to keep the reference stable while allowing selective access by video tokens, and demonstrates improved identity and appearance consistency on an OpenVid‑HD benchmark.

By Ziyang Mai, Yu-Wing Tai