AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,407 stories · RSS feed

arXiv Machine Learning

Jun 3

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606. 03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text.

By Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

llms rag fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 3

FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance

arXiv:2606. 03184v1 Announce Type: cross Abstract: Financial forecasting is difficult due to low signal-to-noise ratios, latent factors, heavy tails, regime shifts, and jumps.

By Jiaze Sun, Kelvin J. L. Koa, Ruiyang Ni, Yize Liu, Haonan Chen, Ke-Wei Huang

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 3

Auditing Engagement Incentives in the Kidfluencer Ecosystem: A Multimodal Weak Supervision Approach

arXiv:2606. 03173v1 Announce Type: cross Abstract: The rise of `kidfluencers' on YouTube has raised ethical concerns about child digital labor and exploitation.

By Zijing Wei, Chao Peter Yang, Xuanjie Chen

llms multimodal safety

arXiv Machine Learning

Jun 3

A Fast Screening Approach for High-dimensional Outcomes and High-dimensional Predictors

arXiv:2606. 03018v1 Announce Type: cross Abstract: Modeling interactions among multimodal, high-dimensional data is intrinsically challenging due to ultra-high dimensionality and complex dependence structure with high level noise.

By Hongju Park, Zhenyao Ye, Shuo Chen

multimodal safety

arXiv Machine Learning

Jun 3

Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observations

arXiv:2606. 03936v1 Announce Type: new Abstract: Neural operator surrogates (NO) approximate PDE solutions orders of magnitude faster than numerical solvers, but suffer from spectral bias: high-frequency content is systematically attenuated, limiting reliability where fine-scale structure matters.

By Niccol\`o Perrone, Fanny Lehmann, Stefania Fresca, Filippo Gatti

diffusion safety

arXiv Machine Learning

Jun 3

Contrastive Neural Algorithmic Reasoning for Graph Coloring

arXiv:2606. 03923v1 Announce Type: new Abstract: Graph coloring seeks to assigns colors to a graph's nodes so that adjacent nodes receive different colors, using as few colors as possible.

By Thien Le, Tianyu Zhao, Melanie Weber

rag safety

arXiv Machine Learning

Jun 3

Easy-to-Use Shielding for Reinforcement Learning

arXiv:2606. 03804v1 Announce Type: new Abstract: Safe exploration is a key challenge in Reinforcement Learning (RL) that aims to prevent agents from making harmful decisions while exploring their environment.

By Stefan Pranger, Bettina K\"onighofer

agents reinforcement-learning safety

arXiv Machine Learning

Jun 3

Spatial Transcriptomics-Guided Alignment Enhances Molecular Profiling in Pathology Foundation Model

arXiv:2606. 03644v1 Announce Type: new Abstract: Comprehensive molecular profiling is essential for modern precision oncology but remains hindered by prohibitive costs, specimen exhaustion, and protracted turnaround times.

By Fengtao Zhou, Yingxue Xu, Zhengyu Zhang, Yihui Wang, Zhengrui Guo, Ling Liang, Jiabo Ma, Cheng Jin, Ziyi Liu, Huajun Zhou, Hongyi Wang, Du Cai, Chenglong Zhao, Xi Wang, Can Yang, Yu Wang, Wenbin Li, Feng Gao, Zhe Wang, Zhenhui Li, Xiuming Zhang, Li Liang, Hao Chen

fine-tuning multimodal safety

arXiv Machine Learning

Jun 3

The Impact of Temporal Granularity on Socio-Demographic Inference from Household Load Profiles

arXiv:2606. 03358v1 Announce Type: new Abstract: Smart meter data can reveal sensitive socio-demographic characteristics of households, raising privacy concerns.

By Dejan Radovanovic, Maximilian Schirl, Andreas Unterweger, G\"unther Eibl

rag safety

arXiv Machine Learning

Jun 3

Validation-Gated Multi-Agent Governance for Online Adaptation of Thermal-Hydraulic Surrogate Models under Operating-Regime Shift

arXiv:2606. 03321v1 Announce Type: new Abstract: Artificial-intelligence surrogates can support second-by-second thermal-hydraulic forecasting, but models selected and frozen offline may become condition-locked once deployed outside their pretraining envelope.

By Doyeong Lim, Seungyoon Lee, In Cheol Bang

llms agents safety

arXiv Machine Learning

Jun 3

Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning

arXiv:2606. 03234v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has become the dominant approach for improving mathematical reasoning in large language models, yet current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.

By Ziyue Wang, Aomufei Yuan, Yongfu Zhu, Shuai Dong, Wenpu Liu, Yiran Yao, Weichu Xie, Yuqi Xu, Caoyuan Ma, Wenqi Shao, Xiaoying Zhang, Nan Duan, Jiaqi Wang

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 3

Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation

arXiv:2606. 03130v1 Announce Type: new Abstract: Small open-source code models that power IDE autocomplete still emit hallucinated Fill-in-the-Middle (FIM) completions: syntactically natural calls to methods, parameters, variables, and imports that do not exist in the surrounding project.

By Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg, Amabel Gale, Xiaoyu Liu, Pareesa Ameneh Golnari, Shengyu Fu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 3

FederatedSkill: Federated Learning for Agentic Skill Evolution

arXiv:2606. 03143v1 Announce Type: new Abstract: Modern LLM agents increasingly rely on skill libraries to handle complex tasks, making skill evolution a primary driver of self-improvement.

By Jingbo Yang, Guanyu Yao, Yang Zhang, Ramana Rao Kompella, Gaowen Liu, Shiyu Chang

llms agents safety

arXiv Machine Learning

Jun 3

Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching

arXiv:2606. 03199v1 Announce Type: new Abstract: Organic crystal structure prediction (CSP) is a requirement for computational modelling of organic solids, but traditionally costs several CPU-years per molecule.

By Alston Lo, Luka Mucko, Austin H. Cheng, Andy Cai, Alastair J. A. Price, Wojciech Matusik, Al\'an Aspuru-Guzik

diffusion benchmarks safety

arXiv Machine Learning

Jun 3

Gate AI: LLM Security Benchmark Evaluation Methodology and Results

arXiv:2606. 02959v1 Announce Type: new Abstract: Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points.

By Ryle Goehausen, Marcus Sousa

llms benchmarks safety

arXiv Machine Learning

Jun 3

Outsmarting the Chameleon: Counterfactual Decoupling for Tactical OOD Shifts in Live Streaming Risk Assessment

arXiv:2606. 02946v1 Announce Type: new Abstract: Live streaming has emerged as a primary medium for social interaction and digital commerce, yet it is increasingly plagued by sophisticated risks.

By Yiran Qiao, Jing Chen, Jiaqi Xu, Yang Liu, Qiwei Zhong, Xiang Ao

benchmarks safety

arXiv Machine Learning

Jun 3

Learning Coherent Representations: A Topological Approach to Interpretability

arXiv:2606. 02841v1 Announce Type: new Abstract: Deep neural networks learn representations where individual features often lack interpretable meaning; a single neuron may activate for scattered, unrelated inputs.

By Sigurd Gaukstad, Melvin Vaupel, Valdemar Karg{\aa}rd Olsen, Erik Hermansen, Benjamin Dunn

rag efficiency safety

arXiv Machine Learning

Jun 3

Making Brain-Computer Interfaces More Secure

arXiv:2606. 02597v1 Announce Type: new Abstract: The development of brain-computer interfaces (BCIs) based on electroencephalograms (EEGs) has advanced significantly mainly to machine learning.

By Md Fahimul Kabir Chowdhury, Gahangir Hossain

safety

arXiv Machine Learning

Jun 3

Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs

arXiv:2606. 02628v1 Announce Type: new Abstract: We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest.

By Aizierjiang Aiersilan

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 3

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

arXiv:2606. 02596v1 Announce Type: new Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections).

By Anherutowa Calvo

llms benchmarks safety

Get the daily briefing