arXiv Computer Vision

Zero-Shot Object Removal via Attention Masking, Latent Anchoring, and Refinement

This paper presents a zero‑shot framework for removing objects from real images using a frozen Stable Diffusion model, avoiding any task‑specific training. The pipeline combines SAM‑based mask construction, BLIP caption conditioning, DDIM inversion, background‑weighted masked null‑text optimization, decoder self‑attention masking, hard outside‑mask latent anchoring, and localized renoise‑denoise refinement. Experiments show effective removal of objects and context‑consistent replacement, with background‑weighted NTI especially helpful for complex backgrounds and repeated refinement reducing residual artifacts.

Hugging Face Trending Papers
Sep 8

MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation

MARS-CLIP is a zero‑shot semantic segmentation framework that builds on CLIP by adding a multi‑resolution feature extraction module and an attention refinement mechanism. The multi‑resolution module fuses fine‑grained local features with global context to mitigate low spatial resolution, while the attention refinement injects spatial and color biases from intermediate layers into the final self‑attention block to better recover object boundaries. Experiments on six public datasets show that MARS‑CLIP outperforms state‑of‑the‑art methods.

arXiv AI
2d ago

Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength

Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength introduces SoftPaint, a zero‑shot sampling method that uses soft masks to provide continuous, pixel‑level control over edits in diffusion models. The approach employs a Langevin‑iteration sampler that respects per‑pixel mask strengths, enabling smooth edits from preserving to fully re‑synthesizing content across image and video backbones. SoftPaint is gradient‑free, memory‑efficient, and works universally with existing diffusion models.

By Candi Zheng, Yuan Lan
arXiv Computer Vision
Aug 28

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

The paper introduces a zero‑shot video restoration and enhancement framework that leverages a text‑to‑image latent diffusion model along with multi‑modal references. It employs dual prompt tuning inversion and sampling to cut inference time to about one‑third of the original, while also strengthening performance and temporal consistency. Additional techniques such as texture‑aware video token merging, referenced self‑attention, and referenced token merging further improve temporal coherence across frames.

By Cong Cao, Huanjing Yue, Xin Liu, Jingyu Yang
arXiv AI
Jun 29

OSOR: One-Step Diffusion Inpainting for Effect-Aware Object Removal

arXiv:2606. 28094v1 Announce Type: cross Abstract: Real-world object removal is challenging due to two key difficulties: the target object's non-local effects, such as shadows and reflections, which are difficult to model, and the fact that user-provided masks are often inaccurate or incomplete.

By Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang
Hugging Face Trending Papers
Jun 9

Don't waste SAM

Meta AI has recently released the Segment Anything Model (SAM), which demonstrates exceptional zero-shot image segmentation performance across various tasks with remarkable accuracy. Despite its inability to provide accurate segmentation across multiple research fields, SAM still serves as a valuable starting point for supporting the segmentation pipeline process, particularly for tasks that require extensive and senior skills annotations.

arXiv Computer Vision
Aug 21

Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal

arXiv:2608. 20212v1 Announce Type: new Abstract: High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections.

By Radim Spetlik, David Futschik, Radek Danecek, Feitong Tan, Ziqian Bai, Rohit Pandey, Yinda Zhang
arXiv AI
Sep 18

CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Models

CleanVideo introduces a selective erasure framework for text-to-video diffusion models, addressing the challenge of removing undesired visual concepts from videos. The method uses a low-dimensional subspace intervention guided by a tri-modal gating mechanism that jointly considers spatiotemporal visual features, timestep signals, and textual semantics to decide where, when, and whether to intervene. Experiments on three video diffusion models demonstrate that CleanVideo effectively erases target concepts while preserving visual fidelity, temporal coherence, and outperforming existing baselines in both frame-level and video-level evaluations, even under concept-recovery attacks.

By Junchi Liao, Hongji Li, Wenrui Zhou, Lijie Hu