arXiv AI August 24, 2026 By Rujin Liang, Zhongpu Chen, Yuhao Lei, Xin Miao

Vis-Poison: Poisoning Visual Knowledge in Multimodal Retrieval-Augmented Generation

Vis-Poison is a novel attack that poisons multimodal retrieval-augmented generation systems by inserting attacker-controlled images as visual evidence, without altering any textual metadata. The attack uses an automated multi-agent approach to create visually plausible poisoned images and has been tested on two multimodal RAG pipelines, four embedding models, and six generation models. In black-box settings, Vis-Poison achieves an end-to-end success rate between 40.16% and 65.40% against 30,000-entry knowledge bases, and remains effective against various multimodal large language models with an average success rate above 60%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision

5d ago

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

arXiv:2511.18921v2 Announce Type: replace Abstract: Backdoor attacks undermine the reliability and trustworthiness of machine learning systems by injecting hidden behaviors that can be maliciously ac...

By Juncheng Li, Yige Li, Hanxun Huang, Yunhao Chen, Xin Wang, Yixu Wang, Xingjun Ma, Yu-Gang Jiang

llms nlp multimodal benchmarks safety

arXiv Machine Learning

Aug 14

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

arXiv:2604. 05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word triggers, namely words that can naturally occur in ordinary textual inputs.

By Yiyang Zhang, Chaojian Yu, Ziming Hong, Yuanjie Shao, Qinmu Peng, Tongliang Liu, Xinge You

nlp multimodal safety

arXiv Machine Learning

Jun 10

MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents

arXiv:2606. 10742v1 Announce Type: cross Abstract: External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences.

By Yv Zhang, Hao Sun, Hao Fang, Kuofeng Gao, Fan Mo, Bin Chen, Shu-Tao Xia, Yaowei Wang

llms agents computer-vision multimodal safety

arXiv AI

Aug 12

Covert Visual Prompt Injection against Commercial Multimodal Large Language Models

arXiv:2603. 29418v2 Announce Type: replace-cross Abstract: Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks.

By Meiwen Ding, Song Xia, Chenqi Kong, Xudong Jiang

llms multimodal safety

arXiv Machine Learning

Jul 20

Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

arXiv:2607. 15657v1 Announce Type: cross Abstract: Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes.

By Halima Bouzidi, Mboutidem Ekemini Mkpong, Mohammad Abdullah Al Faruque

llms agents multimodal safety

arXiv AI

Aug 19

COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models

The paper introduces COMIC, a reference‑aware safety gate designed for multimodal large language models (MLLMs). COMIC detects the operation requested by a user, identifies visual targets through OCR and open‑vocabulary proposals, and evaluates safety on explicit operation‑target pairs, using max‑risk aggregation and quality‑aware routing to decide whether to allow or block a request. Experiments on several open‑source MLLMs and jailbreak benchmarks show that COMIC improves robustness while maintaining benign utility and efficiency.

By Md Abdullahil Oaphy, Anhao Xiang, Zongxing Xie, Huayue Gu, Chenyu Wang, Honghui Xu

llms computer-vision multimodal benchmarks safety

One email a morning, machine-written

Related stories

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents

Covert Visual Prompt Injection against Commercial Multimodal Large Language Models

Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models