arXiv Machine Learning August 6, 2026 By Yiming Chen, Kemou Li, Haiwei Wu, Jiantao Zhou

When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning

Read the original on arXiv Machine Learning →

arXiv:2608. 04052v1 Announce Type: cross Abstract: Backdoor attacks in multimodal contrastive learning (MCL) have garnered growing attention in recent years, as many downstream tasks critically depend on pre-trained MCL models.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

multimodal

arXiv Machine Learning

5d ago

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

arXiv:2604. 05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word triggers, namely words that can naturally occur in ordinary textual inputs.

By Yiyang Zhang, Chaojian Yu, Ziming Hong, Yuanjie Shao, Qinmu Peng, Tongliang Liu, Xinge You

nlp multimodal safety

arXiv AI

Jul 2

Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection

arXiv:2606. 25375v2 Announce Type: replace-cross Abstract: With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud.

By Ching-Hao Chiu, Hao-Wei Chung, Gelei Xu, Xueyang Li, Pin-Yu Chen, John Kheir, Meysam Ghaffari, Carlos Morato, Ahmed Abbasi, Yiyu Shi

llms multimodal benchmarks

Hugging Face Trending Papers

Jun 2

Investigating Adversarial Robustness of Multi-modal Large Language Models

Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder (e. g.

llms rag fine-tuning multimodal safety

arXiv AI

Jun 15

MirrorCheck: Efficient Adversarial Defense for Vision-Language Models

arXiv:2406. 09250v5 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly susceptible to sophisticated adversarial attacks, including adaptive strategies specifically designed to bypass existing defenses.

By Samar Fares, Klea Ziu, Toluwani Aremu, Nikita Durasov, Martin Tak\'a\v{c}, Pascal Fua, Ivan Laptev, Karthik Nandakumar

llms rag diffusion multimodal safety

arXiv AI

Aug 5

Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs

arXiv:2608. 00076v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) increasingly support high-stakes decision making by combining complementary information from images and text.

By Vahidin Hasic, Chao Wang, Luis C. Garcia-Peraza-Herrera, David Watson, Senka Krivic

llms diffusion multimodal benchmarks safety

arXiv Machine Learning

Jun 3

BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks

arXiv:2606. 02947v1 Announce Type: new Abstract: Supervised fine-tuning is the predominant approach for adapting autoregressive vision-language models to downstream tasks.

By Ivan Saboli\'c, Marin Or\v{s}i\'c, Josip \v{S}ari\'c, Sven Lon\v{c}ari\'c

llms fine-tuning multimodal

Related stories

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection

Investigating Adversarial Robustness of Multi-modal Large Language Models

MirrorCheck: Efficient Adversarial Defense for Vision-Language Models

Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs

BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks