The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.
By Tianshi Wang, Jingsong Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu
arXiv:2610.07514v1 Announce Type: new
Abstract: A vision-language model may need to combine an image with a prompt to recognize a safety risk that neither reveals alone. Where does this joint safety...
By Faezeh Dehghan Tarzjani, Mevan Wijewardena, Alexander Romanus, Sampad Mohanty
A$^2$Safe is a framework for safe and effective Visual Question Answering that aligns counterfactual evidence with adaptive agent collaboration. It uses a Grounded Safety Evidence Board to make safety decisions explicit, enforcing invariance to safety‑irrelevant changes while allowing appropriate transitions when risk‑critical evidence changes. The system achieves a 95.72 SIUO safety score, reduces benign refusals on MOSSBench to 14.67%, and maintains a 78.34 average VQA score with 27.8% token overhead.
By Quanxing Xu, Ling Zhou, Xian Zhong, Jinyu Tian, Xiaohua Huang, Rubing Huang, Chia-Wen Lin
The paper introduces Mind2Web-Injection, a benchmark of 9,954 instruction‑screenshot pairs designed to evaluate whether vision‑language models (VLMs) use visual evidence when judging conflicts between on‑screen text and user instructions. Across six VLMs, models with similar overall precision can differ dramatically in Evidence‑Aligned Detection (EAD), revealing that verdict‑only metrics miss critical grounding failures. The authors also propose two training‑free interventions—ReadGate and CmdCompare—to improve evidence grounding and instruction‑command consistency, and argue for separate reporting of verdict correctness, evidence localization, and counterfactual responsiveness.
By Suyoung Lee, Myungsub Choi
arXiv:2607. 00218v1 Announce Type: cross Abstract: Vision-language models (VLMs) are now proposed as runtime safety guards for embodied agents in homes and factories.
By Siddhant Panpatil, Arth Singh, Mijin Koo, Chaeyun Kim, Haon Park, Dasol Choi
arXiv:2607. 22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and personal reputation.
By Yi-Zhi Wang, Yichen Xiao, Linan Yue, Weibo Gao, Yichao Du, Pengfei Fang, Shimin Di, Min-Ling Zhang
arXiv:2606. 02443v1 Announce Type: cross Abstract: Between the first visible sign of danger and the moment an accident occurs, there is often a window where intervention remains possible.
By Yusong Zhao, Yuejin Xie, Youliang Yuan, Junjie Hu, Jitian Guo, Yujiu Yang, Pinjia He
SafeAtlas-VL introduces a large multimodal safety dataset with 1.5 million instances, rating image, request, and response risks on a five‑level ordinal scale across 15 harm categories and 55 subcategories. The accompanying SafeAtlas‑Bench provides 5,000 held‑out cases for evaluating ordinal predictions and continuous risk scores. Models trained on this data, including an 8B Guard model, achieve state‑of‑the‑art performance, outperforming prior benchmarks by about 4% in F1 score.
By Zongrui Wang, Xiangyang Zhu, Sicheng Wang, Han Wang, Dingyi Rong, Zeyu Zhang, Chunyi Li, Yue Shi, Kaiwei Zhang, Zicheng Zhang, Yuan Tian, Qi Jia, Yan Teng, Wei Sun, Ning Liu, Guangtao Zhai
The paper introduces MPS-Bench, a benchmark of 5,181 scenarios from 584 real-world images across 12 high-risk domains, each paired with a hidden user profile, to evaluate personalized safety in vision‑language models (VLMs). Eight leading VLMs were tested and found to almost always respond directly (86‑99%) without seeking missing context, scoring no higher than 2.6/5 on personalized safety. The authors identify a phenomenon called visual dominance, where visual information enters text representations early and suppresses textual risk signals, and propose PRISM, a lightweight input monitor that predicts when a query should be deferred, achieving 0.978 AUC and outperforming all tested models on the safety‑utility Pareto frontier.
By Edward Sun, Yuchen Wu, Zixian Ma, Eric Hanchen Jiang, Yijia Xiao, Xiaoyuan Yi, Ranjay Krishna, Wei Wang, Jindong Wang, Aylin Caliskan
arXiv:2610.09517v1 Announce Type: cross
Abstract: Frozen vision-language models increasingly provide safety signals for reinforcement learning. Their use assumes that similarity to language describin...
By Samuel Tetteh, Cody Fleming
arXiv:2608. 04509v1 Announce Type: new Abstract: Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer.
By De Jiang, Zhengyang Zhang, Kehong Yuan, Shaohua Ma
arXiv:2608.21928v1 Announce Type: new
Abstract: In embodied AI, safety risk can be latent: a benign instruction and a safe scene become hazardous only when composed. Prior work has advanced embodied...
By Zhesheng Zhang, Jiahao Lu, Wei Liu, Cong Pan, Jianhua Yang, Yixiang Chen, Hongyuan Yu, Mengqi Zhang, Kailin Lyu, Zhumin Chen, Keji He