AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,632 stories · RSS feed

arXiv Machine Learning

Jun 25

Do Thinking Tokens Help with Safety?

arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.

By Narutatsu Ri, Abhishek Panigrahi, Sanjeev Arora

llms benchmarks safety

arXiv Machine Learning

Jun 25

Multi-Stream Temporal Fusion for Financial Fraud Detection

arXiv:2606. 25007v1 Announce Type: new Abstract: Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams -- transactions, login sessions, risk signals -- that individually appear benign but collectively reveal fraudulent patterns.

By Mohammadamin Dashti Moghaddam, Nick Sciarrilli

llms benchmarks safety

arXiv Machine Learning

Jun 25

Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis

arXiv:2606. 25606v1 Announce Type: cross Abstract: Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention.

By Felipe Moreno, Sharifa Alghowinem, Hae Won Park, Cynthia Breazeal

fine-tuning benchmarks safety

arXiv Machine Learning

Jun 25

Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets

arXiv:2606. 25760v1 Announce Type: new Abstract: Computer-use agents turn vision-language model (VLM) predictions into executable GUI clicks, so reliable uncertainty estimates are essential for rejection, calibration, miss-severity ranking, and spatial safety regions.

By Divake Kumar, Sina Tayebati, Devashri Naik, Amanda Sofie Rios, Nilesh Ahuja, Omesh Tickoo, Ranganath Krishnan, Amit Ranjan Trivedi

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jun 25

Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models

arXiv:2606. 24952v1 Announce Type: cross Abstract: A central aspiration of mechanistic interpretability is controllability: if we know where a behavior is represented in a model's activations, we should be able to modify it.

By Cosimo Galeone, Anna Ettorre, Minsu Park, Giuseppe Ettorre, Daniele Ligorio

llms fine-tuning safety

arXiv Machine Learning

Jun 25

Benchmarking the Alignment of Data-Quality Metrics, Human Judgment and Land-Cover Segmentation Performance for Earth Observation

arXiv:2606. 25128v1 Announce Type: cross Abstract: Volume and quality of datasets are crucial for deep learning model training, yet they are often constrained by availability and data acquisition costs.

By \"Umit Mert \c{C}a\u{g}lar, Alptekin Temizel

computer-vision benchmarks safety

arXiv Machine Learning

Jun 25

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

arXiv:2606. 25627v1 Announce Type: new Abstract: Distributed intelligent systems increasingly need to train across data silos without centralizing raw data.

By Erdenebileg Batbaatar, Young Yoon

fine-tuning safety

arXiv Machine Learning

Jun 25

KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Tagging

arXiv:2512. 07420v3 Announce Type: replace-cross Abstract: Jet identification plays a central role in analyzing data from high-energy collider experiments.

By Md Raqibul Islam, Adrita Khan, Mir Sazzat Hossain, Choudhury Ben Yamin Siddiqui, Md. Zakir Hossan, Tanjib Khan, M. Arshad Momen, Amin Ahsan Ali, AKM Mahbubur Rahman

benchmarks safety

arXiv Machine Learning

Jun 25

Safe Learning Control with Optimality and Stability Guarantees

arXiv:2501. 15373v2 Announce Type: replace-cross Abstract: Merely pursuing performance may adversely affect safety, while a conservative policy for safe exploration will degrade the performance.

By Xinyang Wang, Hongwei Zhang, Shimin Wang, Wei Xiao, Martin Guay

reinforcement-learning robotics safety

arXiv Machine Learning

Jun 25

Kuramoto Oscillatory Phase Encoding: Neuro-inspired Synchronization for Improved Learning Efficiency

arXiv:2604. 07904v2 Announce Type: replace Abstract: Spatiotemporal neural dynamics and oscillatory synchronization are widely implicated in biological information processing and have been hypothesized to support flexible coordination such as feature binding.

By Mingqing Xiao, Yansen Wang, Dongqi Han, Caihua Shan, Dongsheng Li

llms computer-vision benchmarks safety

arXiv Machine Learning

Jun 25

Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See

arXiv:2606. 25127v1 Announce Type: new Abstract: We investigate how reward design shapes the internal attention patterns of reinforcement learning agents trained for autonomous driving.

By Mohamed Benabdelouahad, Ahmed Djalal Hacini, Nadir Farhi, Aissa Boulmerka

agents reinforcement-learning robotics safety

arXiv Machine Learning

Jun 25

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

arXiv:2606. 25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisfying multiple safety constraints.

By Ankur Naskar, Swetha Ganesh, Vaneet Aggarwal

agents reinforcement-learning safety

arXiv Machine Learning

Jun 25

Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory

arXiv:2606. 23195v2 Announce Type: replace Abstract: Large Language Model (LLM) agents increasingly rely on memory systems to maintain long-term coherence.

By Zewen Liu

llms agents safety

arXiv Machine Learning

Jun 25

Statistically Valid Hyperparameter Selection: From Tuning to Guarantees

arXiv:2606. 25601v1 Announce Type: cross Abstract: Hyperparameter selection is a critical step in the deployment of modern artificial intelligence systems, given the need to tune degrees of freedom such as inference-time parameters, implementation-level settings, and thresholds driving decision rules.

By Amirmohammad Farzaneh, Osvaldo Simeone

safety

arXiv Machine Learning

Jun 25

When Do Conservation Laws Survive Learned Representations? Certified Horizons for Latent World Models

arXiv:2606. 24945v1 Announce Type: new Abstract: We ask a representation-learning question about physical world models: when does a conservation law remain certifiable after a model learns a latent representation?

By Hongbo Wang

safety

arXiv Machine Learning

Jun 25

ASAP: Agent-System Co-Design for Wall-Clock-Centered Auto HPO Research for ML Experiments

arXiv:2606. 25207v1 Announce Type: new Abstract: Hyperparameter Optimization (HPO) is essential for maximizing machine learning model performance, and its core challenge is sample efficiency: finding strong configurations within a limited budget.

By Taicheng Guo, Haomin Zhuang, Kehan Guo, Yujun Zhou, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang

llms agents safety

arXiv Machine Learning

Jun 25

Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion

arXiv:2606. 25097v1 Announce Type: new Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs?

By Sahil Kadadekar

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 25

Leaking Circuit Secrets: Gradient Leakage Attacks on Graph Neural Networks

arXiv:2606. 25589v1 Announce Type: new Abstract: As graph neural networks (GNNs) become standard tools for critical tasks in circuit design and analysis, their security and privacy risks require careful attention.

By Rupesh Raj Karn, Johann Knechtel, Ozgur Sinanoglu

efficiency benchmarks safety

arXiv Machine Learning

Jun 25

Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model

arXiv:2507. 09052v3 Announce Type: replace-cross Abstract: Training data for class-conditional image synthesis often exhibit a long-tailed distribution with limited amount of images for tail classes.

By Fang Chen, Alex Villa, Gongbo Liang, Fuxing Li, Xiaoyi Lu, Meng Tang

llms diffusion safety

arXiv Machine Learning

Jun 25

TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting

arXiv:2606. 25439v1 Announce Type: new Abstract: Deep learning-based models have achieved state-of-the-art performance in Time Series Forecasting (TSF), yet their evaluation remains dominated by pointwise error metrics such as Mean Squared Error (MSE), which quantify numerical accuracy but overlook structural properties of the forecast signal, including recurrent dynamics, oscillatory behavior, and phase alignment.

By Sandeepa Weerasekara, Sandareka Wickramanayake

llms rag benchmarks safety

Get the daily briefing