AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI

Oct 7

SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy

arXiv:2609.15039v3 Announce Type: replace-cross Abstract: User prompts provided to large language models (LLMs) may contain private information. One way to protect them is to execute the LLM inside a...

By Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar, Dali Kaafar

llms efficiency safety

arXiv AI

Oct 7

Cross-Lingual Activation Steering for Multilingual Language Models

arXiv:2601.16390v2 Announce Type: replace-cross Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...

By Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar

llms benchmarks safety

arXiv Machine Learning

Oct 7

A theory of platonic representations in language models

arXiv:2610.07168v1 Announce Type: new Abstract: Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic rep...

By Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner, Daniel J. Korchinski, Santiago Acevedo, Matthieu Wyart

llms safety

arXiv Machine Learning

Oct 7

Reward-Driven Learning under Prompt-Level Differential Privacy

arXiv:2610.07212v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model on problems that may themselves be confidential, and the trained model ca...

By Jiachen Zhao, Antonia Januszewicz, Taeho Jung

llms reinforcement-learning fine-tuning safety

arXiv AI

Oct 7

Incentive Alignment in Online Experimentation

arXiv:2610.05922v2 Announce Type: replace-cross Abstract: Evaluating the causal effect of new features is a central goal for online platforms. While recent literature addresses limited testing traffi...

By Ermis Soumalias, Richard Mudd, Abbas Zaidi

agents safety

arXiv Machine Learning

Oct 7

ATLAS-AL: Adaptive Trust-Region for Latent Adversarial Searches via Active Learning

arXiv:2610.07323v1 Announce Type: new Abstract: Security evaluation of learning-based systems requires more than just testing the system against a fixed collection of attacks. It requires adaptive me...

By Marsalis Gibson, Claire Tomlin, Shankar Sastry

safety

arXiv Machine Learning

Oct 7

Multigroup Fairness and Omniprediction: Separations and Equivalences

arXiv:2610.07374v1 Announce Type: new Abstract: Omniprediction is a learning guarantee which requires a single predictor to be competitive relative to the best hypothesis from a benchmark class for a...

By S\'ilvia Casacuberta, Parikshit Gopalan, Varun Kanade, Omer Reingold, Konstantinos Stavropoulos, Pranay Tankala

benchmarks safety

arXiv Machine Learning

Oct 7

Weight Oracles: Reading Neural Network Weights with Language Models

arXiv:2610.07334v1 Announce Type: new Abstract: Interpretability methods for neural networks are predominantly reactive: they analyse activations produced during specific forward passes, requiring kn...

By Krishna Kabra, Constantin Venhoff, Christian Schroeder de Witt

llms fine-tuning safety

arXiv Machine Learning

Oct 7

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

arXiv:2610.07406v1 Announce Type: new Abstract: Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn an...

By Yuta Kobayashi, Divyam Madaan, Shalmali Joshi

safety

arXiv Machine Learning

Oct 7

Interpretable Hypergraph Learning via Neural Additive Models

arXiv:2610.07458v1 Announce Type: new Abstract: Hypergraphs offer a natural framework for modeling networked data, where dependencies among entities are governed by higher-order interactions. While h...

By Shihan Feng, Xin Zheng, Shiyi Yang, Ren Wang, Chudi Zhong, Can Chen

benchmarks safety

arXiv Machine Learning

Oct 7

Adaptive Model Inversion Attacks Generalize a Privacy-Robustness Tradeoff

arXiv:2610.07677v1 Announce Type: new Abstract: In this paper, we show that standard evaluations of high-resolution Model Inversion Attacks (MIAs) significantly underestimate training-data privacy le...

By Shailen Smith, Rasmus Torp, Adam Breuer

benchmarks safety

arXiv Machine Learning

Oct 7

Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures

arXiv:2610.07754v1 Announce Type: new Abstract: Adversarial training is one of the most reliable defenses against adversarial attacks, but its high computational cost must generally be paid anew for...

By Soichiro Kumano

llms safety

arXiv Machine Learning

Oct 7

Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation

arXiv:2610.07786v1 Announce Type: new Abstract: Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete v...

By Donghan He, Luhuan Wu

benchmarks safety

arXiv Machine Learning

Oct 7

Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis

arXiv:2610.07804v1 Announce Type: new Abstract: Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation...

By Martin Eppert, Krishna Balasubramanian, Subhro Ghosh, Jason Klusowski, Yan Shuo Tan

llms safety

arXiv Machine Learning

Oct 7

Retrieval Is Not Enough: Refreshing Memory for Frozen Time-Series Forecasters

arXiv:2610.07834v1 Announce Type: new Abstract: Retrieval-augmented time-series forecasting uses the continuations of historical segments similar to the current context as references for a forecaster...

By Chao He, Jianyu Xu, Xinyi Guo, Ruiqi Liu, Haobin Ding, Ruiqi He, Dongqing Song

rag benchmarks safety

arXiv Machine Learning

Oct 7

Can phenotypic activity be predicted without experimental readouts?

arXiv:2610.07997v1 Announce Type: new Abstract: Molecular encoders contrastively pretrained on paired molecule-morphology data, such as CLOOME and CellCLIP, have been proposed as cheap surrogates for...

By T\'elio Cropsal, Roc\'io Mercado

safety

arXiv Machine Learning

Oct 7

Performative Prediction with Selective Labels

arXiv:2610.08272v1 Announce Type: new Abstract: Many social applications of machine learning exhibit performative effects: population behavior changes in response to deployed models. Performative pre...

By Giovani Valdrighi, Isabel Valera, Marcos Medeiros Raimundo

safety

arXiv Machine Learning

Oct 7

Structure-Aware Graph Abstention for Reliable Selective Forecasting

arXiv:2610.08322v1 Announce Type: new Abstract: Selective forecasting abstains on high-risk test windows under a retained-coverage budget. Existing gates such as TEM (Brusokas et al., 2025) score eac...

By Jianxiang Xie, Belal Alsinglawi

benchmarks safety

arXiv Machine Learning

Oct 7

Random Feature Gaussian Process Attention: Linear-Time Probabilistic Attention with Calibrated Uncertainty

arXiv:2610.08578v1 Announce Type: new Abstract: Transformers provide a state-of-the-art modeling framework, yet poor calibration limits their reliability in safety-critical applications. A promising...

By Amir Mohammad Mahfoozi, Zi Yang, Ying Li, Michael Minyi Zhang

llms benchmarks safety

arXiv Machine Learning

Oct 7

What Does Fr\'echet Distance Measure? A Directional Decomposition

arXiv:2610.05518v1 Announce Type: cross Abstract: The Fr\'echet distance is a de facto standard for evaluating generative models across domains, appearing as FID for images and FVD for videos. It sum...

By Yunghee Lee, Jaeyeon Kim

rag diffusion safety

Get the daily briefing