arXiv Computer Vision

FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment

The paper investigates why vision‑language models (VLMs) often miss hazards, attributing this to an overly conservative decision criterion rather than poor perception. By identifying and steering an affective ‘fear’ circuit within the model, the authors shift the decision threshold, dramatically increasing hazard recall from 27.0 % to 75.7 % without retraining. The study demonstrates that affect‑based inference-time manipulation can correct VLM decision biases in practical applications.

arXiv Computation and Language
Sep 17

When Cognitive Graphs Meet LLMs: BDEI Cognitive Pathways for Panic Emotional Arousal Prediction

The paper introduces PanicCognitivePath (PCP), a model that predicts the timing of panic emotional arousal by integrating appraisal emotion theory into a Belief‑Desire‑Emotion‑Intention (BDEI) framework. PCP uses a Psychological Safety Distance (PSD) model to fuse physical, social, cognitive, and informational signals into a unified risk metric, and confines large language models to a single parameter‑estimation step to reduce hallucinations. Experiments on Hurricane Sandy data show PCP improves individual prediction accuracy by 10.68% and reduces peak count error to 7.07%.

By Mengzhu Liu, Long Qin, Chuan Ai, Zhengqiu Zhu, Hongru Liang, Fangfang Li, Chen Gao, Yong Li, Xin Lu, Quanjun Yin
arXiv Machine Learning
Oct 23

Probing and Steering for Safety-Relevant Behaviours in Large Language Models

The paper introduces activation probes—directions in a language model’s residual stream—that can both detect and steer safety‑relevant behaviors such as deception, high‑stakes awareness, and sycophancy. By evaluating different estimators (difference‑of‑means vs. logistic regression) and token‑pooling schemes (last, mean, learned attention) on Llama‑3.1‑8B‑Instruct, the authors find that logistic regression excels at detection while difference‑of‑means excels at steering, except for sycophancy where the pattern reverses. The study highlights that the optimal pooling method, layer choice, and steering metric vary by behavior and data, and provides exported probes that enable an interactive interface for scoring and steering text.

By Nimun Bajwa, Massimo Poesio
arXiv AI
Jul 3

ESC: Emotional Self-Correction for Reliable Vision-Language Models

arXiv:2607. 02089v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, yet they remain vulnerable to unreliable reasoning.

By Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy, Hung Viet Nguyen, Huy Nguyen Minh Nhat, Thanh-Huy Nguyen, Cuong Tuan Nguyen, Hoang M. Le, Dat Nguyen, Phat Kim Huynh, Min Xu, Ulas Bagci
arXiv AI
Oct 16

Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models

The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.

By Tianshi Wang, Jingsong Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu
arXiv AI
Oct 26

An Agentic Approach with Evidence-Grounded Verification for Multimodal Emotion Reasoning in Conversation

The paper proposes an agentic framework for multimodal emotion reasoning that uses frozen models to avoid costly instruction tuning. It employs modality‑specific perception agents, an integration agent, and an MLLM critic to generate emotion predictions with evidence citations, followed by an evidence‑grounded verification stage that identifies errors and applies targeted prompt patches without updating model weights. On the MER2025 dataset, this approach improves F‑score across several large language‑vision models, outperforming static pipelines and matching or exceeding reflection baselines.

By Elyas Rashno, Farhana Zulkernine
arXiv AI
Sep 7

When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs

The paper introduces MPS-Bench, a benchmark of 5,181 scenarios from 584 real-world images across 12 high-risk domains, each paired with a hidden user profile, to evaluate personalized safety in vision‑language models (VLMs). Eight leading VLMs were tested and found to almost always respond directly (86‑99%) without seeking missing context, scoring no higher than 2.6/5 on personalized safety. The authors identify a phenomenon called visual dominance, where visual information enters text representations early and suppresses textual risk signals, and propose PRISM, a lightweight input monitor that predicts when a query should be deferred, achieving 0.978 AUC and outperforming all tested models on the safety‑utility Pareto frontier.

By Edward Sun, Yuchen Wu, Zixian Ma, Eric Hanchen Jiang, Yijia Xiao, Xiaoyuan Yi, Ranjay Krishna, Wei Wang, Jindong Wang, Aylin Caliskan