arXiv AI
From Local Evidence to Safety Verdicts: Causal Tracing in Vision-Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 25
EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models
arXiv:2608.23313v1 Announce Type: new Abstract: Vision-language model safety benchmarks typically evaluate only final responses: whether a model refuses, warns, or complies. This outcome-level view c...
arXiv AI
2d ago
Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models
The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.
arXiv AI
Sep 3
Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
The paper investigates cross‑modal safety drift in multimodal large language models, where a harmless text query paired with a visual image can trigger harmful responses. Empirical analysis identifies unsafe response patterns and shows that visual cues receive limited attention, weakening refusal mechanisms. The authors introduce Safety‑Awareness Representation Transfer (SRT), a lightweight method that transfers safety signals from text processing to mitigate cross‑modal drift while maintaining model utility.
arXiv AI
Aug 25
GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
arXiv:2608.21928v1 Announce Type: new Abstract: In embodied AI, safety risk can be latent: a benign instruction and a safe scene become hazardous only when composed. Prior work has advanced embodied...
arXiv AI
Aug 6
CARGO-VL: Counterfactual Arbitration with Risk-Constrained Group Optimization for Vision-Language Models
arXiv:2608. 04509v1 Announce Type: new Abstract: Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer.
arXiv AI
Sep 7
When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs
The paper introduces MPS-Bench, a benchmark of 5,181 scenarios from 584 real-world images across 12 high-risk domains, each paired with a hidden user profile, to evaluate personalized safety in vision‑language models (VLMs). Eight leading VLMs were tested and found to almost always respond directly (86‑99%) without seeking missing context, scoring no higher than 2.6/5 on personalized safety. The authors identify a phenomenon called visual dominance, where visual information enters text representations early and suppresses textual risk signals, and propose PRISM, a lightweight input monitor that predicts when a query should be deferred, achieving 0.978 AUC and outperforming all tested models on the safety‑utility Pareto frontier.