arXiv AI June 18, 2026 By Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He

Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation

arXiv:2511. 20002v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 30

High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

arXiv:2512. 21815v4 Announce Type: replace-cross Abstract: Vision-language models (VLMs) achieve remarkable performance but remain vulnerable to adversarial attacks.

By Mengqi He, Xinyu Tian, Xin Shen, Jinhong Ni, Shu Zou, Zhaoyuan Yang, Jing Zhang

llms multimodal safety

arXiv AI

Jun 16

Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades

arXiv:2606. 15308v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) have shown strong visual reasoning abilities, serving a large model for every query is computationally expensive.

By Zhongye Liu, Yaopei Zeng, Yurui Chang, Lu Lin

llms multimodal safety

arXiv Machine Learning

Jun 5

Steering Vectors are an Adversarial Attack Surface

arXiv:2606. 05958v1 Announce Type: new Abstract: Activation steering has become a popular way to control Large Language Model (LLM) behavior without fine-tuning.

By Abzal Aidakhmetov, Donato Crisostomi, Tommaso Mencattini, Adrian Robert Minut, Iacopo Masi, Emanuele Rodol\`a

llms fine-tuning multimodal safety

arXiv AI

Jun 24

PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

arXiv:2606. 24388v1 Announce Type: new Abstract: We introduce a large-scale, open-source dataset of pre-generated adversarial attacks for vision-language models (VLMs).

By Simone Gallivanone, Hossein Khodadadi, Mauro Dore, Mauro Medda, Nicola Franco

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

6d ago

LLM Router: Rethinking Routing with Prefill Activations

arXiv:2603. 20895v3 Announce Type: replace-cross Abstract: Existing routers rely on semantic query features or handcrafted features, which often fail to capture model-specific failures or intrinsic task difficulty.

By Tanay Varshney, Annie Surla, Michelle Xu, Gomathy Venkata Krishnan, Maximilian Jeblick, David Austin, Neal Vaidya, Davide Onofrio

llms

arXiv Machine Learning

Aug 11

Preference Redirection via Attention Concentration: An Attack on Computer Use Agents

arXiv:2604. 08005v2 Announce Type: replace Abstract: Advancements in multimodal foundation models have enabled the development of Computer Use Agents (CUAs) capable of autonomously interacting with GUI environments.

By Dominik Seip, Matthias Hein

agents fine-tuning multimodal safety

Related stories

High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades

Steering Vectors are an Adversarial Attack Surface

PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

LLM Router: Rethinking Routing with Prefill Activations

Preference Redirection via Attention Concentration: An Attack on Computer Use Agents