AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning

Sep 22

Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents

arXiv:2609.22510v1 Announce Type: cross Abstract: As LLM applications integrate with external tools, they are increasingly exposed to indirect prompt injection (IPI), where adversarial instructions a...

By Justin Szczepaniak, Elad Feldman, Naum Viner, Ben Nassi

llms agents benchmarks safety

arXiv Machine Learning

Sep 22

CLEAR: Complex Learned Explicit Analytical Regularization for Ultra-Accelerated 4D Flow CMR Reconstruction

arXiv:2609.22950v1 Announce Type: cross Abstract: While compressed-sensing regularizers enable interpretable reconstruction of 4D Flow CMR through transparent variational objectives, their hand-craft...

By German Sh\^ama Wache, Sebastian Neumayer

benchmarks safety

arXiv Machine Learning

Sep 22

Topographic Training Concentrates Causal Circuits Without Improving Neuron Monosemanticity

arXiv:2609.24379v1 Announce Type: cross Abstract: Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entan...

By Gautam Ranka, Shubham Santosh Pandere, Aiden Dsouza

llms efficiency safety

arXiv Machine Learning

Sep 22

Beyond Point Prediction: Artificial Representative Trees with Uncertainty

arXiv:2609.24528v1 Announce Type: cross Abstract: Random forests (RFs) predict well but are opaque, whereas single decision trees are interpretable but unstable. Artificial representative trees (ARTs...

By Lea L. Mairh\"ofer, Silke Szymczak, Bj\"orn-Hergen Laabs, Tuwe L\"ofstr\"om-Cavallin

benchmarks safety

arXiv Machine Learning

Sep 22

D-JEPA: A Decision-Aligned Latent World Model

arXiv:2609.24749v1 Announce Type: cross Abstract: Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate wil...

By Shuaijun Liu, Chengyu Wu, Qifu Wen, Feiyang You, Chenglong Zhang, Shuyang Hao, Xi Lin, Ningxin Su

agents robotics safety

arXiv Machine Learning

Sep 22

XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts

arXiv:2609.24770v1 Announce Type: cross Abstract: Localising artifacts in synthetic speech remains challenging, as most evaluation methods yield only global quality scores. This paper presents XSQ-AS...

By Ben Heritage, Luca Resti, M\'onica Villanueva Aylagas, Timothy Mehlenbacher, Konrad Tollmar, James Alfred Walker

llms safety

arXiv Machine Learning

Sep 22

onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

arXiv:2609.24983v1 Announce Type: cross Abstract: We present onPanda, an interactive tool for efficiently annotating LLM alignment data and agent trajectories. onPanda adopts token-level correction a...

By Lei Yang, Mengyin Liu, Jia Wang, Hangyu Guo, Liang Zhao, Zheng Ge, Kang An, Binxing Jiao, Qi Han, Daxin Jiang, Siqi Shen, Xiangyu Zhang

llms agents fine-tuning benchmarks safety

arXiv Machine Learning

Sep 22

Disassociating performance from compositional feature learning

arXiv:2505.09716v4 Announce Type: replace Abstract: Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and...

By George Dimitriadis, Spyridon Samothrakis

llms benchmarks safety

arXiv Machine Learning

Sep 22

Glass-Box Deep Learning for FDIA Detection in Nonlinear Automatic Generation Control: A Kolmogorov-Arnold Network Approach

arXiv:2509.05259v2 Announce Type: replace Abstract: Automatic Generation Control (AGC) plays a critical role in maintaining power balance across multi-area power systems. However, its complete relian...

By Ahmad Mohammad Saber, Alok Paranjape, Jehad Jilan, Niranjana Naveen Nambiar, Amr Youssef, Deepa Kundur

fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Sep 22

Implementation of Quantum Implicit Neural Representation in Deterministic and Probabilistic Autoencoders for Image Reconstruction/Generation Tasks

arXiv:2603.06755v3 Announce Type: replace Abstract: We propose a quantum implicit neural representation (QINR)-based autoencoder (AE) and variational autoencoder (VAE) for image reconstruction and ge...

By Saadet M\"uzehher Eren

diffusion safety

arXiv Machine Learning

Sep 22

When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneity

arXiv:2605.08992v2 Announce Type: replace Abstract: Federated learning (FL) is increasingly used to fine-tune foundation models (FMs) on distributed private data. The community largely assumes that l...

By Kiran Naseer, Umar Shoaib

nlp fine-tuning safety

arXiv Machine Learning

Sep 22

How Can Incentives and Cut Layer Selection Influence Data Contribution in Split Federated Learning?

arXiv:2412.07813v4 Announce Type: replace-cross Abstract: To alleviate the training burden in federated learning while enhancing convergence speed, Split Federated Learning (SFL) has emerged as a pro...

By Joohyung Lee, Jungchan Cho, Wonjun Lee, Mohamed Seif, H. Vincent Poor

safety

arXiv Machine Learning

Sep 22

Attack-Resistant Uniform Fairness for Linear and Smooth Contextual Bandits

arXiv:2602.04125v2 Announce Type: replace-cross Abstract: Modern digital platforms use contextual bandits to allocate valuable exposure and opportunities among competing participants. Fair treatment...

By Qingwen Zhang, Wenjia Wang

safety

arXiv Machine Learning

Sep 22

On Kernel Eigen-alignments of KRR: Reconstruction and Generalization

arXiv:2605.15240v2 Announce Type: replace-cross Abstract: This paper investigates the critical role of eigenalignments between the kernel matrix and learning targets in achieving robust generalizatio...

By Yang Liu, Ernest Fokoue, Richard Lange, Daniel Krutz

safety

arXiv Machine Learning

Sep 22

TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling

arXiv:2605.27690v2 Announce Type: replace-cross Abstract: LLM agents increasingly operate through multi-turn tool use and environment interaction, where safety risks often emerge from intermediate st...

By Jiaqian Li, Yanshu Li, Boxuan Zhang, Ruixiang Tang, Kuan-Hao Huang

llms agents benchmarks safety

arXiv Machine Learning

Sep 22

Constant Swap Regret in General-Sum Games via Two-Scale Higher-Order Optimism

arXiv:2609.16751v2 Announce Type: replace-cross Abstract: We give deterministic and uncoupled learning dynamics for finite multiplayer general-sum games under full-information feedback that achieve c...

By Tung Mai

reinforcement-learning safety

arXiv Computation and Language

Sep 22

Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents

arXiv:2609.22090v1 Announce Type: new Abstract: An LLM producing the response pattern associated with a human psychological effect is not the same claim as the LLM possessing that bias. We present Ps...

By Joy Bose

llms agents robotics benchmarks safety

arXiv Computation and Language

Sep 22

Team DArgk at the 2026 ELOQUENT lab for evaluating generative language model quality: Residuals of Humanity: AI Detection Evasion via GRPO Fine-Tuning

arXiv:2609.22221v1 Announce Type: new Abstract: Large language models (LLMs) can generate fluent and coherent text that is increasingly difficult to distinguish from human writing, motivating the dev...

By Antonela Tommasel, Juan Manuel Rodriguez

llms reinforcement-learning fine-tuning safety

arXiv Computation and Language

Sep 22

Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations

arXiv:2609.22255v1 Announce Type: new Abstract: Existing approaches to persona simulation with Large Language Models (LLMs) mostly rely on shallow character descriptions that fail to sustain coherent...

By Rotem Dror, Zohar Elyoseph, Yuval Haber, Elad Refoua, Oshrat Ayalon, Adir Solomon

llms agents benchmarks safety

arXiv Computation and Language

Sep 22

Toward Personalized Sleep Guidance from Wearable Data Using Language Models

arXiv:2609.22463v1 Announce Type: new Abstract: Sleep monitoring using wearable data has shown promise for personal health, yet large language model (LLM)-based summarization and question answering r...

By Yusheng Tan, Running Zhao, Sofia Angel, Ninghui Hao, Ash Arian, Nikita N. Dulin, Jay Lin, Ou Zhu, Faiza Shaik, Xinxing Yang, Bonnie W. Leung, Katie Roster, Arlene Ruiz de Luzuriaga, Kenneth Lee, Alejandra Lastra, Habibul Ahsan, Guihong Wan

llms agents nlp fine-tuning safety

Get the daily briefing