Hugging Face Trending Papers

Zero-Shot Object Removal via Attention Masking, Latent Anchoring, and Refinement

arXiv Computer Vision
Sep 24

Zero-Shot Object Removal via Attention Masking, Latent Anchoring, and Refinement

This paper presents a zero‑shot framework for removing objects from real images using a frozen Stable Diffusion model, avoiding any task‑specific training. The pipeline combines SAM‑based mask construction, BLIP caption conditioning, DDIM inversion, background‑weighted masked null‑text optimization, decoder self‑attention masking, hard outside‑mask latent anchoring, and localized renoise‑denoise refinement. Experiments show effective removal of objects and context‑consistent replacement, with background‑weighted NTI especially helpful for complex backgrounds and repeated refinement reducing residual artifacts.

By Arman Taghizadeh (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Ulf Krumnack (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Kai-Uwe K\"uhnberger (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany)
arXiv AI
2d ago

Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength

Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength introduces SoftPaint, a zero‑shot sampling method that uses soft masks to provide continuous, pixel‑level control over edits in diffusion models. The approach employs a Langevin‑iteration sampler that respects per‑pixel mask strengths, enabling smooth edits from preserving to fully re‑synthesizing content across image and video backbones. SoftPaint is gradient‑free, memory‑efficient, and works universally with existing diffusion models.

By Candi Zheng, Yuan Lan
Hugging Face Trending Papers
Sep 8

MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation

MARS-CLIP is a zero‑shot semantic segmentation framework that builds on CLIP by adding a multi‑resolution feature extraction module and an attention refinement mechanism. The multi‑resolution module fuses fine‑grained local features with global context to mitigate low spatial resolution, while the attention refinement injects spatial and color biases from intermediate layers into the final self‑attention block to better recover object boundaries. Experiments on six public datasets show that MARS‑CLIP outperforms state‑of‑the‑art methods.

arXiv Computer Vision
Aug 28

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

The paper introduces a zero‑shot video restoration and enhancement framework that leverages a text‑to‑image latent diffusion model along with multi‑modal references. It employs dual prompt tuning inversion and sampling to cut inference time to about one‑third of the original, while also strengthening performance and temporal consistency. Additional techniques such as texture‑aware video token merging, referenced self‑attention, and referenced token merging further improve temporal coherence across frames.

By Cong Cao, Huanjing Yue, Xin Liu, Jingyu Yang
arXiv AI
Jun 29

OSOR: One-Step Diffusion Inpainting for Effect-Aware Object Removal

arXiv:2606. 28094v1 Announce Type: cross Abstract: Real-world object removal is challenging due to two key difficulties: the target object's non-local effects, such as shadows and reflections, which are difficult to model, and the fact that user-provided masks are often inaccurate or incomplete.

By Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang
arXiv Computer Vision
Aug 21

Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal

arXiv:2608. 20212v1 Announce Type: new Abstract: High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections.

By Radim Spetlik, David Futschik, Radek Danecek, Feitong Tan, Ziqian Bai, Rohit Pandey, Yinda Zhang
arXiv Computer Vision
Sep 3

SR-Edit: Region-Aware Image Editing via Self-Refinement

SR-Edit is a new image editing framework that uses iterative self‑refinement to improve fidelity. At each step it extracts precise, self‑consistent region separations from the model’s predictions and then enforces preservation in non‑edit areas with correction updates that stay aligned with the original sampling dynamics. Experiments show that SR‑Edit delivers better preservation and overall image quality than existing editing techniques.

By Andong Wang, Zehua Chen, Yuxuan Jiang, Jun Zhu
Hugging Face Trending Papers
Jun 9

Don't waste SAM

Meta AI has recently released the Segment Anything Model (SAM), which demonstrates exceptional zero-shot image segmentation performance across various tasks with remarkable accuracy. Despite its inability to provide accurate segmentation across multiple research fields, SAM still serves as a valuable starting point for supporting the segmentation pipeline process, particularly for tasks that require extensive and senior skills annotations.