AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Machine Learning

Sep 1

Compression-Aware Abstention: Teaching LLMs to Refuse When KV-Compression Masks Remove Answer Evidence

arXiv:2608.29934v1 Announce Type: cross Abstract: KV-cache compression reduces LLM inference memory by evicting context tokens, but when the evicted tokens contain answer-bearing evidence, the model...

By Mohammadali Khodabandehlou, Bhaskar Krishnamachari

llms fine-tuning safety

arXiv Machine Learning

Sep 1

When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs

arXiv:2608.29936v1 Announce Type: cross Abstract: Safety alignment of large language models (LLMs) degrades across languages, yet the internal mechanism driving this asymmetry remains poorly understo...

By Apoorva Upadhyaya, Sandipan Sikdar

llms safety

arXiv Machine Learning

Sep 1

Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment

arXiv:2608.29978v1 Announce Type: cross Abstract: Large language models are increasingly required to generate responses that satisfy multiple competing objectives. Since optimal trade-offs depend on...

By Lingxiao Kong, Steffen Staab, Cong Yang, Oya Beyan, Zeyd Boukhers

llms safety

arXiv Machine Learning

Sep 1

A Hybrid State-Space Approach for Census-Tract Population Estimation

arXiv:2608.30094v1 Announce Type: cross Abstract: Sequence models---the architecture family behind large language models and, increasingly, state-of-the-art image recognition---have redefined how mac...

By Jackson R. Ye, Alexandre V. Morozov

llms computer-vision benchmarks safety

arXiv Machine Learning

Sep 1

Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization

arXiv:2608.30141v1 Announce Type: cross Abstract: Preference optimization is widely used to align large language models with human preferences, but preference-data composition may also influence priv...

By Dishu Yang, Jingjing Liu, Jize Li

llms reinforcement-learning efficiency safety

arXiv Machine Learning

Sep 1

Fairness in multi-class multi-group classification problems via contextial coherent risk measures

arXiv:2608.30223v1 Announce Type: cross Abstract: We propose a new design of fair classifiers for multi-class classification problems in the presence of vector-valued sensitive attributes. In that sc...

By Darinka Dentcheva, Xiangyu Tian

safety

arXiv Machine Learning

Sep 1

Learning the Geometry of Admissible Hypotheses through Inductive Bias in Training Distributions

arXiv:2608.31028v1 Announce Type: cross Abstract: Scientific discovery often requires reasoning over competing hypotheses that are consistent with experimental observations. For mixed-variable and co...

By James Crowley, Faez Ahmed, Anton van Beek

rag efficiency benchmarks safety

arXiv Machine Learning

Sep 1

Segmentation of Bovid Dentition Under Imperfect Annotations: A Comparative Study of Convolutional and Attention Models

arXiv:2608.31052v1 Announce Type: cross Abstract: Semantic segmentation decomposes an image into distinct mask regions corresponding to different object categories, such as people, cars, signs or bui...

By Keith G. Mills, Evan B. Sanders, Gregory J. Matthews, Juliet K. Brophy

llms computer-vision safety

arXiv Machine Learning

Sep 1

Multi-Marginal Flow Matching with Adversarially Learnt Interpolants

arXiv:2510.01159v3 Announce Type: replace Abstract: Learning the dynamics of a process given sampled observations at several time points is an important but difficult task in many scientific applicat...

By Oskar Kviman, Kirill Tamogashev, Nicola Branchini, V\'ictor Elvira, Jens Lagergren, Esmeralda S. Whitammer

diffusion safety

arXiv Machine Learning

Sep 1

The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning

arXiv:2511.21799v2 Announce Type: replace Abstract: Real-world machine learning (ML) pipelines rarely produce a single model; instead, they produce a Rashomon set of many near-optimal ones. We show t...

By Ethan Hsu, Harry Chen, Chudi Zhong, Lesia Semenova

safety

arXiv Machine Learning

Sep 1

Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs

arXiv:2601.03087v2 Announce Type: replace Abstract: Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box LLM...

By David Hartmann, Lena Pohlmann, Lelia Hanslik, Noah Gie{\ss}ing, Bettina Berendt, Pieter Delobelle

llms safety

arXiv Machine Learning

Sep 1

Universal Redundancies in Time Series Foundation Models

arXiv:2602.01605v2 Announce Type: replace Abstract: Time Series Foundation Models (TSFMs) leverage extensive pretraining to accurately predict unseen time series during inference, without the need fo...

By Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai, William Gilpin

llms fine-tuning benchmarks safety

arXiv Machine Learning

Sep 1

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

arXiv:2603.02482v2 Announce Type: replace Abstract: Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds a...

By Zhongxi Wang, Yueqian Lin, Jingyang Zhang, Zinuo Cheng, Hai Helen Li, Yiran Chen

llms multimodal safety

arXiv Machine Learning

Sep 1

SYNAPSE: Neuro-Symbolic Visual Thought-to-Text Decoding via Topological Semantic Denoising

arXiv:2605.27790v2 Announce Type: replace Abstract: Recent advances in large language models have accelerated open-vocabulary EEG-to-imagined-text decoding, where non-invasive neural activity recorde...

By Akshaj Murhekar, Abhijit Mishra

llms fine-tuning benchmarks safety

arXiv Machine Learning

Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain

llms reinforcement-learning fine-tuning safety