arXiv AI

Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing

The paper introduces WaterBird-Edit, a benchmark that separates bird habitat labels from background context to detect evidence misgrounding in multimodal knowledge editing. It proposes VICE, a retrieval‑based framework that stores edits in external memory and uses visual in‑context demonstrations to focus on task‑relevant evidence. Experiments on three benchmarks show VICE improves lifelong editing performance and reduces sensitivity to background changes on WaterBird-Edit.

Hugging Face Trending Papers
Jul 23

PC-Edit: Prompt-Contrastive Region Discovery and Region-Guided Editing

Replacing an object with one that differs in category or shape requires complete source removal, natural target formation unconstrained by the source silhouette, and preservation of unrelated content. Existing training-free editors either localize edits from terminal predictions under source and target prompts or preserve unrelated content through spatially unselective source-feature reuse without explicit region discovery.

arXiv Computer Vision
Oct 9

Reasoning-Informed Visual Editing

The paper introduces RISEBench and its expanded version RISEBench++, a benchmark for evaluating reasoning-informed visual editing (RISE). RISEBench++ categorizes tasks into six reasoning dimensions—Temporal, Causal, Spatial, Logical, Counterfactual, and Hybrid—decomposed into 12 subcategories and 65 fine-grained task types, and includes 1,000 human‑annotated test cases in English and Chinese. The authors also present RISE‑Agent, a training‑free framework that integrates reasoning‑driven planning, tool‑augmented execution, and verifier‑guided refinement, outperforming most existing approaches on diverse RISE tasks.

By Xue Yang, Peiyuan Zhang, Yilun Zhu, Qihao Yang, Mingxin Liu, Xiangyu Zhao, Ziqian Fan, Zhaokai Wang, Yan Li, Yifan Yang, Xu Yang, Xiaosong Jia, Yue Zhou, Zhihang Zhong, Junchi Yan
arXiv AI
Aug 25

ReasonEdit: Editing Vision-Language Models using Human Reasoning

ReasonEdit is a new editor for vision‑language models that allows users to provide reasoning explanations during the editing process. It stores human reasoning in a codebook and retrieves relevant facts at inference time using a topology‑balanced multimodal embedding approach inspired by network science. Experiments on four VLMs and multiple rationale‑based visual question answering datasets show that incorporating human reasoning leads to state‑of‑the‑art editing performance and better generalization.

By Jiaxing Qiu, Kaihua Hou, Roxana Daneshjou, Ahmed Alaa, Thomas Hartvigsen
arXiv AI
Oct 12

The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs

The paper investigates how visual interventions—training‑free edits applied to multimodal large language models—affect hallucinations and prediction quality. By applying a visual‑residual edit across successive decoder layers and matching its RMS strength, the authors identify an Evidence Anchoring Boundary (EAB) where the intervention’s influence drops sharply; effects before the EAB are 13.78–26.10× larger than after it. They further compare counterfactual subtraction across different question types and introduce calibrated selective routing, which only alters predictions when calibration supports an intervention, improving attribute accuracy by 1.56–1.81 percentage points while avoiding the harms of universal subtraction.

By Qiji Zhou, Junyi Hu, Peiyu Wang, Yue Zhang