arXiv AI August 19, 2026 By Yang Chen, Zhan Zhuang, Yanbin Wei, Zebin Chen, Hua Liu, Yu Zhang

Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models

The paper introduces ADAPT, an adversarial disentangled prompt tuning framework designed to improve the robustness of vision‑language models. ADAPT employs a dual‑prompt strategy: a target prompt learns robust features while a set of decoy prompts capture pseudo‑robust, non‑generalizable shortcuts. By enforcing orthogonality between target and decoy prompts, the method mitigates robust generalization overfitting and provides a theoretical error bound for unseen classes, leading to significant empirical robustness gains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 8

SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models

arXiv:2606. 06943v1 Announce Type: cross Abstract: Vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition but remain highly fragile under adversarial perturbations.

By Sunoh Kim, Daeho Um

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 13

Robustifying Vision-Language Models via Test-Time Prompt Adaptation

arXiv:2607. 09450v1 Announce Type: cross Abstract: Pre-trained Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under adversarial perturbations.

By Xingyu Zhu, Huanshen Wu, Shuo Wang, Beier Zhu, Jiannan Ge, Jiaheng Zhang, Long Chen

llms multimodal safety

Hugging Face Trending Papers

Jun 2

Investigating Adversarial Robustness of Multi-modal Large Language Models

Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder (e. g.

llms rag fine-tuning multimodal safety

arXiv Machine Learning

Jun 16

Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization

arXiv:2507. 02288v2 Announce Type: replace-cross Abstract: Domain Generalization (DG) seeks to develop a versatile model capable of performing effectively on unseen target domains.

By De Cheng, Zhipeng Xu, Xinyang Jiang, Dongsheng Li, Nannan Wang, Xinbo Gao

llms benchmarks safety

arXiv AI

Jul 22

Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model

arXiv:2607. 18958v1 Announce Type: cross Abstract: While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks, posing significant security risks.

By Sibo Wang, Jie Zhang, Shiguang Shan, Xilin Chen, Wen Gao

llms nlp fine-tuning multimodal benchmarks safety

arXiv AI

Jul 7

A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning

arXiv:2607. 03600v1 Announce Type: cross Abstract: Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains.

By Sushant Dagaji Desale, Rahul Mishra, Ashutosh Kumar Sinha

reinforcement-learning fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models

Robustifying Vision-Language Models via Test-Time Prompt Adaptation

Investigating Adversarial Robustness of Multi-modal Large Language Models

Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization

Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model

A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning