AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,825 stories · RSS feed

arXiv Machine Learning

Jun 5

Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoders

arXiv:2509. 22015v2 Announce Type: replace Abstract: Standard Sparse Autoencoders (SAEs) excel at discovering a dictionary of a model's learned features, providing a powerful lens for passive feature discovery.

By Jianrong Ding, Muxi Chen, Chenchen Zhao, Qiang Xu

safety

arXiv Machine Learning

Jun 5

FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting

arXiv:2408. 11336v3 Announce Type: replace Abstract: Climate change stands as one of the most pressing global challenges of the twenty-first century, with far-reaching consequences such as rising sea levels, melting glaciers, and increasingly extreme weather patterns.

By Tajamul Ashraf, Janibul Bashir

llms benchmarks safety

arXiv Machine Learning

Jun 5

Grounded but Misleading: Evaluating Semantic Alignment in AI-Generated Security Explanations

arXiv:2602. 05056v2 Announce Type: replace-cross Abstract: Online scams increasingly leverage fluent and context-aware social engineering strategies, creating growing demand for AI systems that explain why a message may be risky.

By Heajun An, Connor Ng, Sandesh Sharma Dulal, Junghwan Kim, Jin-Hee Cho

llms safety

arXiv Machine Learning

Jun 5

GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

arXiv:2606. 05860v1 Announce Type: new Abstract: Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise.

By Oleeviya Babu Poikarayil, C\'edric Schockaert, Abdulrahman Nahhas, Christian Daase, Mursal Dawodi, Jawid Ahmad Baktash

llms agents efficiency benchmarks safety

arXiv Machine Learning

Jun 5

Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models

arXiv:2511. 21338v2 Announce Type: replace Abstract: Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in principle, should enable more uniform context utilisation.

By Julianna Piskorz, Cristina Pinneri, Alvaro Correia, Motasem Alfarra, Risheek Garrepalli, Christos Louizos

llms diffusion fine-tuning safety

arXiv Machine Learning

Jun 5

CASS-RTL: Correctness-Aware Subspace Steering for RTL Generation with LLMs

arXiv:2606. 05680v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled the automatic synthesis (generation) of register-transfer level (RTL) code from natural language instructions, offering a promising pathway to accelerate chip design.

By Mohammad Akyash, Nowfel Mashnoor, Kimia Azar, Hadi Kamali

llms rag agents fine-tuning safety

arXiv Machine Learning

Jun 5

Electricity price forecasting across Norway's five bidding zones in the post-crisis era

arXiv:2604. 26634v2 Announce Type: replace Abstract: Norway's electricity market is heavily dominated by hydropower, but the 2021-2022 energy crisis and stronger integration with Continental Europe have fundamentally altered price formation, reducing the reliability of forecasting models calibrated on historical data.

By My Thi Diem Phan, Trung Tuyen Truong, Hoai Phuong Ha, Dat Thanh Nguyen

multimodal benchmarks safety

arXiv Machine Learning

Jun 5

LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges

arXiv:2605. 10807v4 Announce Type: replace-cross Abstract: The integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) and hardware security is rapidly reshaping the semiconductor industry.

By Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri

llms agents benchmarks safety

arXiv Machine Learning

Jun 5

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

arXiv:2605. 08318v2 Announce Type: replace Abstract: We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators.

By Brandon Yee, Pairie Koh, Jack Rodriguez, Mihir Tekal

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 5

PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysis

arXiv:2606. 06293v1 Announce Type: new Abstract: Whilst the vulnerability of graph neural networks (GNNs) to adversarial attacks poses a critical threat to graph representation learning, the understanding of the robust generalization behavior remains a fundamental challenge in the adversarial setting.

By Ziling Liang, Xinping Yi, Qingsong Wen, Shi Jin

safety

arXiv Machine Learning

Jun 5

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

arXiv:2505. 05026v5 Announce Type: replace-cross Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept.

By Jaehyun Jeon, Min Soo Kim, Jang Han Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, Youngjae Yu

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 5

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability

arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.

By Yash Aggarwal, Atmika Gorti, Vinija Jain, Aman Chadha, Krishnaprasad Thirunarayan, Manas Gaur

llms efficiency safety

arXiv Machine Learning

Jun 5

Expand Neurons, Not Parameters

arXiv:2510. 04500v3 Announce Type: replace Abstract: This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance.

By Linghao Kong, Inimai Subramanian, Yonadav Shavit, Micah Adler, Dan Alistarh, Nir Shavit

rag safety

arXiv Machine Learning

Jun 5

Consistency Training Along the Transformer Stack

arXiv:2606. 05817v1 Announce Type: new Abstract: Consistency training encourages models to behave similarly across different contexts, and has shown promise for reducing misalignment.

By Sukrati Gautam, Neil Shah, Arav Dhoot, Bryan Maruyama, Caroline Wei, Rohan Kapoor, Robert Sidey, Prakhar Gupta, Zi Cheng Huang, David Demitri Africa

llms safety

arXiv Machine Learning

Jun 5

Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models

arXiv:2606. 05434v1 Announce Type: new Abstract: Group Relative Policy Optimisation (GRPO) has emerged as an effective reinforcement-learning algorithm for aligning language models on reasoning tasks, but it treats every token position and every sampled rollout symmetrically.

By Chirag Chawla, Rohan Charudatt Salvi, Madhav S. Baidya

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 5

SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks

arXiv:2606. 05609v1 Announce Type: cross Abstract: As large language models (LLMs) are widely deployed, identifying their vulnerability through jailbreak attacks becomes increasingly critical.

By Seungwon Jeong, Jiwoo Jeong, Hyeonjin Kim, Yunseok Lee, Woojin Lee

llms multimodal safety

arXiv Machine Learning

Jun 5

Generalized TV--$\ell_p$ Structured Priors for Bayesian $T_1$ Mapping

arXiv:2606. 05381v1 Announce Type: new Abstract: We propose an extended family of structured spatial priors that incorporates the total variation (TV) function with $\ell_p$ norms.

By Disi Lin, Martin Berggren, Tommy L\"ofstedt

rag safety

arXiv Machine Learning

Jun 5

ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

arXiv:2606. 05170v1 Announce Type: new Abstract: At matched accuracy, open-weight LLMs differ substantially in the shape of their error severity distribution -- a difference invisible to the scalar error rate.

By Jason Z Wang

llms benchmarks safety

arXiv Machine Learning

Jun 5

Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation

arXiv:2606. 05403v1 Announce Type: new Abstract: Language models increasingly act as epistemic proxies, synthesizing evidence from multiple sources to inform decisions.

By Rohan N. Pradhan, Steve Goley

llms safety

arXiv Machine Learning

Jun 5

High-Dimensional Theory of LoRA Fine-Tuning in a Solvable Attention Model

arXiv:2606. 05899v1 Announce Type: new Abstract: We develop a high-dimensional statistical theory of low-rank adaptation (LoRA) in attention models, capturing the interplay between pre-training and fine-tuning.

By O. Duranthon, F. Boncoraglio, L. Zdeborov\'a

fine-tuning safety

Get the daily briefing