AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 16

Same Answer, Different Representations: Hidden instability in VLMs

arXiv:2602.06652v2 Announce Type: replace Abstract: The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions r...

By Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema, Wai-Chung Kwan, Fazl Barez, Maria Sofia Bucarelli, Fabrizio Silvestri, Pasquale Minervini
arXiv Machine Learning
Sep 16

Crash Narrative-Guided Countermeasure Recommendation Using Large Language Models: A Retrieval-Augmented Generation Framework for Intersection Safety

arXiv:2609.15997v1 Announce Type: cross Abstract: Improving safety at intersections requires identifying crash mechanisms and recommending appropriate countermeasures. However, this process tradition...

By Abu Saif Md Nasim Uddin, Mohamed Abdel-Aty, Zubayer Islam, Parvez Anowar, Chenzhu Wang
arXiv Machine Learning
Sep 16

Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits

Meta-LinEXP3 is an online-within-online algorithm designed for adversarial linear contextual bandits with random action sets. It builds a task-level prior from completed tasks to guide an inner LinEXP3 learner, achieving an σO(√n) per‑task regret when context distributions are known and an σO(n^{2/3}) regret with a past‑only regularized moment estimator when they are unknown. The paper also links prior accuracy to transfer regret, showing that better priors yield sublinear, transfer‑dependent regret across tasks, and demonstrates the method on structured hyperspectral tensor sampling.

By Hao Li, Jie Xu, Zheng Xie
arXiv AI
Sep 16

Differential privacy representation geometry for medical image analysis

The paper introduces Differential Privacy Representation Geometry for Medical Imaging (DP‑RGMI), a framework that interprets differential privacy as a structured transformation of representation space. DP‑RGMI decomposes performance loss into encoder geometry—measured by representation displacement and spectral effective dimension—and task‑head utilization, quantified by the gap between linear‑probe and end‑to‑end utility. Across 594,000 chest X‑ray images from four datasets, the study finds that differential privacy consistently creates a utilization gap even when linear separability remains, while displacement and spectral dimension vary non‑monotonically with initialization and dataset, indicating that privacy alters representation anisotropy rather than uniformly collapsing features.

By Soroosh Tayebi Arasteh, Marziyeh Mohammadi, Sven Nebelung, Daniel Truhn
arXiv AI
Sep 16

Evaluating Prompt Robustness in Text-to-Audio Systems for Adaptive Virtual Agents and Game Soundtracks

The paper evaluates how robust three text‑to‑audio models—MusicGen‑small, MusicGen‑large, and Stable Audio 2.5—are to small changes in prompts that could affect adaptive game soundtracks. Using metrics such as log‑Mel distance, MFCC/chroma‑DTW, and CLAP similarity, the study finds that Stable Audio 2.5 consistently yields the lowest acoustic distances and highest CLAP similarity when prompts are structurally rephrased, while MusicGen‑large performs best under lexical substitutions and intensity shifts. The authors also observe that Stable Audio 2.5 shows the greatest variation in prompt‑to‑audio alignment across different random seeds, highlighting the need for multi‑seed robustness testing in game audio applications.

By Jiahui Wu, Mei Si
arXiv AI
Sep 16

Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation

The paper investigates how large language models (LLMs) generate distractor answers for multiple‑choice questions (MCQs) by modeling student misconceptions. It introduces a learning‑science‑based taxonomy of reasoning strategies and applies it to LLM‑generated reasoning traces in math and science MCQs. The study finds that in math, LLMs often follow a misconception‑based process that can be diagnostically useful, whereas in science they rely more on semantic similarity, with frequent failures when the model cannot produce a correct solution or discards plausible distractors. Providing the correct solution in the prompt improves alignment with human distractors by 6.4%. "whyItMatters":"The findings show that anchoring distractor generation to the correct solution enhances LLM alignment with human‑authored distractors, underscoring the importance of correct‑answer cues in educational AI."

By Yanick Zengaffinen, Andreas Opedal, Donya Rooein, Kv Aditya Srivatsa, Shashank Sonkar, Mrinmaya Sachan
arXiv Computer Vision
Sep 16

ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models

The paper introduces ViD, a vision‑dominant gender bias mitigation framework for large vision‑language models. ViD uses causal analysis of attention patterns and dual mechanisms—backdoor adjustment and refined token selection—to suppress bias while preserving reasoning and generation quality. Experiments show a 14.7% reduction in gender bias on FACET and significant improvements on MS COCO image captioning, all without extra training overhead.

By Zhipeng Zhao, Zhaoqiang Wei, Peishun Liu, Youwei Zhao, Ruichun Tang
arXiv AI
Sep 16

Affect-Prototype Guided Fusion for Open-Vocabulary Incomplete Multi-modal Emotion Recognition

The paper introduces Affect-Prototype Guided Fusion (APCF), a framework for open‑vocabulary multimodal emotion recognition that handles incomplete and unsynchronized modal data. APCF builds an affect‑prototype library to model how different emotions contribute across modalities, enabling dynamic fusion of available features. The fused representations are then decoded by an LLM to generate open‑vocabulary emotion labels, achieving superior performance on OV‑MERD+ and MER‑FG datasets compared to existing methods.

By Yichi Zhang, Shenyue Wang, Jing Luo, Chunyang Yu, Xinyu Yang
arXiv Computer Vision
Sep 16

Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models

The paper examines how the tight integration of multimodal understanding and image generation in Unified Multimodal Models (UMMs) can create new safety vulnerabilities. It introduces RICE, an attack framework that exploits bidirectional interactions between generation and understanding to propagate unsafe signals across modalities. Experiments demonstrate high attack success rates in both Generation‑to‑Understanding and Understanding‑to‑Generation pathways, revealing previously overlooked safety weaknesses in UMMs.

By Kaishen Wang, Heng Huang