← Back to all news
arXiv AI By Chao Huang, Meng Tong, Kejiang Chen

Fingerprinting Multimodal Large Language Models

Read the original on arXiv AI →

The paper introduces AttnPrint, a white‑box fingerprinting method that extracts low‑frequency components of cross‑modal attention distributions to identify multimodal large language models (MLLMs). It also presents DistillTrace, a black‑box auditing tool that uses hypothesis testing of MLLM outputs to detect potential model infringement. Experiments on 154 model instances across 19 architectures show that AttnPrint effectively detects derivative models and remains robust to downstream modifications, while DistillTrace reveals distillation relationships under various techniques.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 24

Vis-Poison: Poisoning Visual Knowledge in Multimodal Retrieval-Augmented Generation

Vis-Poison is a novel attack that poisons multimodal retrieval-augmented generation systems by inserting attacker-controlled images as visual evidence, without altering any textual metadata. The attack uses an automated multi-agent approach to create visually plausible poisoned images and has been tested on two multimodal RAG pipelines, four embedding models, and six generation models. In black-box settings, Vis-Poison achieves an end-to-end success rate between 40.16% and 65.40% against 30,000-entry knowledge bases, and remains effective against various multimodal large language models with an average success rate above 60%.

By Rujin Liang, Zhongpu Chen, Yuhao Lei, Xin Miao
llmsragagentsmultimodal
More like this →
arXiv AI
Aug 26

Seeing vs. Believing: Evaluating the Language Bias of Open-Source MLLMs in Counter-Intuitive Scenes

The paper introduces CAIT, a benchmark of 400 synthetic scenes featuring counter‑intuitive actions that challenge multimodal large language models (MLLMs). Human participants and proprietary models like Claude and Gemini perform well, but standard open‑source instruction‑tuned MLLMs fail, largely due to a strong language prior that overrides contradictory visual evidence. The study shows that Chain‑of‑Thought reasoning can help but introduces new issues, while targeted fine‑tuning and structured prompting can reduce reliance on language priors and improve visual grounding.

By Chen Ling, Tongwei Zhang, Hanqian Li, Nai Ding
llmsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Aug 14

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

arXiv:2604. 05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word triggers, namely words that can naturally occur in ordinary textual inputs.

By Yiyang Zhang, Chaojian Yu, Ziming Hong, Yuanjie Shao, Qinmu Peng, Tongliang Liu, Xinge You
nlpmultimodalsafety
More like this →
arXiv AI
Aug 24

ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models

ReFrame is a training‑free framework that enhances safety alignment for multimodal large language models at test time. It uses two lightweight agents: one generates risk and utility evidence, and the other rewrites prompts and routes images to create a safe proxy before invoking the deployed MLLM. Experiments show that ReFrame improves jailbreak defense, safety awareness, and reduces over‑sensitivity while maintaining multimodal utility.

By Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai, Dawei Feng, Huaimin Wang
llmsagentsmultimodalbenchmarkssafety
More like this →