AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computer Vision
Sep 15

Restore What Matters: Lessons from Joint Restoration and Recognition

arXiv:2609.13791v1 Announce Type: new Abstract: Recognition pipelines typically adopt a restore-then-recognize workflow, yet decades of experience show that generating visually pleasing images seldom...

By Lanqing Guo, Xijun Wang, Minchul Kim, Yu Yuan, Wes Robbins, Xingguang Zhang, Nicholas Chimitt, Stanley H. Chan, Zhangyang Wang, Xiaoming Liu
arXiv AI
Sep 15

UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics

UniCAR‑RL is an annotation‑free reinforcement learning framework designed to improve multimodal large language models’ visual mathematics reasoning. It decouples perception and reasoning by using three branches: Caption‑RL for perception optimization, Reasoning‑RL for logical reasoning with a gold image description, and QA‑RL for end‑to‑end question answering. Experiments show significant gains in mathematical and visual reasoning across various model architectures and scales using only raw short‑answer data.

By Yuzhe Li, Hao Yan, Hao Wang, Xingchen Liu, Ya-Qi Yu, Jihao Wu, Minghui Liao, Wei Chen, Yuliang Liu
arXiv AI
Sep 15

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

arXiv:2609.15855v1 Announce Type: cross Abstract: % !TEX root = ../main.tex People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk con...

By Laura M. Vowels, Matthew J. Vowels, Shivali Sharma, Apoorv Jha, Rehnuma Choudhury, Wasseem El Sarraj, Rachel Francois-Walcott, Aruba Hussain, Sarah Ingram, Angela Loulopoulou, Adva Segal, Elena Volkova
arXiv Computer Vision
Sep 15

AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

arXiv:2606.19100v4 Announce Type: replace Abstract: Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open...

By Diogo Gl\'oria-Silva, Jo\~ao Cardeira, Manuel Letras da Luz, Afonso Simpl\'icio, Gon\c{c}alo Vinagre, Diogo Tavares, Rafael Ferreira, In\^es Calvo, In\^es Vieira, David Semedo, Jo\~ao Magalh\~aes
arXiv AI
Sep 15

Generative Interpretability via Scalable Neuro-Symbolic Models

The paper argues that as Large Language Models transition from chatbots to agentic systems, the current post-hoc interpretability paradigm is insufficient for safe deployment because it cannot audit or intervene before an output is produced. It proposes a shift to generative interpretability, where a model’s inference process inherently exposes semantically meaningful checkpoints that are human-understandable and can be causally intervened upon. The authors illustrate the advantages of this approach and introduce Neuro‑Symbolic Models as a concrete implementation.

By Xiaocong Yang
arXiv AI
Sep 15

Evaluation Metrics for Safe Reinforcement Learning

The paper introduces new evaluation metrics for safe reinforcement learning that go beyond average safety guarantees by examining how often and how severely safety bounds are violated, consistency across tasks and bounds, and the relationship between training-time and final policy behavior. It also proposes a safety tier system for categorizing algorithms and presents empirical safety evaluations on multiple navigation tasks. The authors recommend reporting aggregate metrics, distributional data, and task‑specific results together, and provide an open‑source suite, SafeRLEval, to facilitate reliable safety assessment.

By Lindsay Spoor, Aske Plaat, Thomas Moerland