arXiv Machine Learning

It Is Not Seeing the Hazard: A Frozen Vision-Language Safety Score Measures Its Caption Bank

arXiv AI
Aug 12

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

arXiv:2608. 10513v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety alignment inherited from their language backbones.

By Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang Zheng
arXiv AI
Jul 10

AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding

arXiv:2607. 08745v1 Announce Type: new Abstract: Recent advances in Vision-Language Models, Large Language Models, and Multimodal Large Language Models have improved autonomous driving tasks such as scene understanding, decision making, trajectory prediction, and visual question answering.

By Siddharth Damodharan, Radhika Gupta, Ali Alshami, Ryan Rabinowitz, Jugal Kalita
arXiv Machine Learning
2d ago

BD-HazardVLM: Probing Vision-Language Models for Latent Defensive-Driving Hazards in Bangladesh Road Scenes

BD-HazardVLM is a diagnostic benchmark of 500 Bangladesh road-scene images designed to probe vision‑language models’ defensive‑driving reasoning in weakly structured Global South traffic. The benchmark includes annotations for hazard presence, category, risk timing, severity, defensive actions, visible evidence, and bias traps, and evaluates models such as Qwen2.5‑VL‑7B, InternVL3‑8B, and MiniCPM‑V‑4.6. Results show high binary hazard‑presence accuracy but poor fine‑grained reasoning, with category accuracy below 40 %, risk‑timing accuracy under 13 %, and defensive‑action matching under 2 %. whyItMatters":"The study demonstrates that coarse hazard recognition does not guarantee reliable fine‑grained defensive‑driving reasoning, highlighting the need for geographically diverse safety evaluation of vision‑language models."

By Abdullah Al Maruf, Md. Sajedul Islam, Tanmoy Mridha, Irfan Hossain Bhuiyan
arXiv AI
2d ago

Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models

The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.

By Tianshi Wang, Jingsong Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu
arXiv AI
Jun 24

UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving

arXiv:2606. 24759v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal reasoning and spatial precision.

By Xiaowei Gao, Pengxiang Li, Yitai Cheng, Ruihan Xu, James Haworth, Stephen Law, Yun Ye
arXiv Machine Learning
Jun 3

VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring

arXiv:2606. 03954v1 Announce Type: cross Abstract: As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not.

By Hanjiang Hu, Yiyuan Pan, Jiaxing Li, Xusheng Luo, Alexander Robey, Na Li, Yebin Wang, Changliu Liu