arXiv Machine Learning

ROCOCO: Robust Outline Completion of Occluded Curvilinear Objects

arXiv AI
Jun 29

OSOR: One-Step Diffusion Inpainting for Effect-Aware Object Removal

arXiv:2606. 28094v1 Announce Type: cross Abstract: Real-world object removal is challenging due to two key difficulties: the target object's non-local effects, such as shadows and reflections, which are difficult to model, and the fact that user-provided masks are often inaccurate or incomplete.

By Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang
arXiv Computer Vision
Sep 2

PredErase: Training-Free Object-and-Effect Removal with Predictive Latent Guidance

PredErase is a training‑free method that removes objects and their photometric effects by guiding a frozen Fill model with predictive latent cues. It expands the user‑provided mask to a contact‑band region, uses I‑JEPA to generate a context‑conditioned target for the hole, and aligns Fill’s completions with this target while keeping surrounding pixels fixed. On benchmarks such as RemovalBench, RORD‑Val, and DEFACTO‑Val, PredErase improves the native FLUX.2 backbone for instance‑only masks, though supervised removers still outperform it on full‑image metrics.

By Waikit Xiu, Qiang Lu, Junbiao Chen, Xiying Li
arXiv Machine Learning
3d ago

OccJEPA: Occlusion as Masking for Amodal Video Segmentation

OccJEPA treats occlusion as masking for amodal video segmentation by dropping tokens outside an object's visible mask and using a pre‑trained V‑JEPA 2.1 predictor to fill in the missing features, followed by a lightweight head that decodes these into complete masks. The resulting deterministic model is 33.5× faster than diffusion‑based state‑of‑the‑art methods and achieves higher accuracy on SAIL‑VOS, TAO‑Amodal (including a held‑out test split), and Kubric. Experiments show that the performance gains stem from the predictor’s pre‑trained weights rather than its size, with models ranging from 87 M to 1.85 B parameters.

By Genpei Zhang
arXiv Computer Vision
Oct 9

IS-Diff: Improving Diffusion-Based Inpainting with Better Initial Seed

The paper introduces IS‑Diff, a training‑free diffusion‑based inpainting method that improves semantic consistency by using initial seeds sampled from unmasked regions. It incorporates a dynamic selective refinement mechanism to adjust the initialization strength during intermediate latent stages. Experiments on CelebA‑HQ, ImageNet, and Places2 show that IS‑Diff outperforms state‑of‑the‑art methods across standard and large‑mask inpainting tasks.

By Yongzhe Lyu, Yu Wu, Yutian Lin, Bo Du
arXiv Computer Vision
Sep 3

Structured-Prior-Guided Diffusion Inpainting with Physical Consistency for Traffic Sign Augmentation

The paper introduces a structured‑prior‑guided diffusion inpainting framework that enhances traffic sign augmentation by incorporating semantic, appearance, and geometric priors through text prompts, vector templates, and ControlNet. It enforces physical consistency with colour and edge‑structure losses, achieving superior reconstruction fidelity and OCR accuracy compared to larger models. The synthetic data generated improves detection performance for rare traffic sign classes by up to 7.4× over a real‑data‑only baseline.

By Luo Li, Chongchong Huang, Jun Jia, Qiang Gao, Xinlong Liu, Gui Yang, Liang Cao
arXiv Computation and Language
Oct 12

OccludeBench: Can VLMs Find What's Missing?

OccludeBench is a manually curated benchmark that tests vision‑language models (VLMs) on their ability to recover fully occluded objects from remaining visual evidence. The benchmark distinguishes direct cases—requiring recovery from a specific physical cue such as a shadow or reflection—from inferential cases, which rely on broader scene context. Across five VLMs, accuracy averages 42.7 % on direct cases versus 62.7 % on inferential cases, a 20‑point gap, while humans achieve 63.4 % and 68.5 % respectively with no significant gap. The study introduces OccludeTrace, a four‑stage diagnostic that isolates the failure point, revealing that models struggle most when recovering the relevant object from the full image projection. Interventions such as inpaint‑then‑confront reduce the gap to a statistically non‑significant difference for models with image‑generation capabilities.

By Jasin Cekinmez, Addison J. Wu, Ryo Mitsuhashi, Linrong Cai, Xingyu Fu, Zhuang Liu