AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,221 stories · RSS feed

arXiv AI

Jun 8

How reliable are LLMs when it comes to playing dice?

arXiv:2606. 07515v1 Announce Type: cross Abstract: We investigate the probabilistic reasoning capabilities of large language models through a controlled benchmarking study on discrete probability problems.

By Luca Avena, Gianmarco Bet, Bernardo Busoni

llms rag benchmarks safety

arXiv AI

Jun 8

Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization

arXiv:2606. 07033v1 Announce Type: new Abstract: Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events, including categories unseen during training.

By Zhe Yang, Ruyi Zhang, Hongtao Chen, Wenrui Li, Hengyu Man, Wangmeng Zuo, Xiaopeng Fan

multimodal benchmarks safety

arXiv Machine Learning

Jun 8

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models

arXiv:2505. 10892v2 Announce Type: replace Abstract: Post-training LLMs with RLHF and preference optimization methods (e.

By Akhil Agnihotri, Rahul Jain, Deepak Ramachandran, Zheng Wen

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 8

Latent-space Attacks for Refusal Evasion in Language Models

arXiv:2605. 21706v2 Announce Type: replace Abstract: Safety-aligned language models are trained to refuse harmful requests, yet refusal behavior can be suppressed by steering their internal representations.

By Giorgio Piras, Raffaele Mura, Fabio Brau, Maura Pintor, Luca Oneto, Fabio Roli, Battista Biggio

llms multimodal benchmarks safety

arXiv AI

Jun 8

Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach

arXiv:2510. 09041v3 Announce Type: replace-cross Abstract: Deep reinforcement learning (DRL) has demonstrated remarkable success in developing autonomous driving policies.

By Junchao Fan, Qi Wei, Ruichen Zhang, Yang Lu, Jianhua Wang, Xiaolin Chang, Bo Ai

agents reinforcement-learning robotics benchmarks safety

arXiv AI

Jun 8

Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting

arXiv:2603. 02220v2 Announce Type: replace-cross Abstract: Time series forecasting remains a challenging problem due to the intricate entanglement of intra-period fluctuations and inter-period trends.

By Yixin Wang, Yifan Hu, Peiyuan Liu, Naiqi Li, Tao Dai, Shu-Tao Xia

benchmarks safety

arXiv AI

Jun 8

MSAIC-Net: A Multi-Scale Attention and Imbalance-Aware Contrastive Network for ECG-Based Myocardial Substrate Abnormality Detection

arXiv:2606. 06718v1 Announce Type: cross Abstract: Myocardial substrate abnormalities, such as myocardial scar and myocardial infarction (MI), are associated with adverse cardiovascular outcomes.

By Canyu Lei, Fenglin Zhang, Derek Bivona, Cristiane Singulane, Jonathan Pan, Kenneth Bilchick, Amit R. Patel, Jianxin Xie

safety

arXiv AI

Jun 8

Generative Models Erode Human Temporal Learning Through Market Selection

arXiv:2606. 06572v1 Announce Type: cross Abstract: We argue that modern generative models create structural risks for knowledge and cultural production at current, sub-AGI capability levels.

By Wenjun Cao

safety

arXiv AI

Jun 8

Hearing the Unspoken: Language Model Priors for Acoustic Adversarial Attacks

arXiv:2606. 06833v1 Announce Type: cross Abstract: Automatic Speech Recognition (ASR) systems operating in real-time settings must process acoustic input under strict temporal constraints, where transcription decisions are inherently made on incomplete information.

By Jiani Xie, Andrew C. Cullen, Paul Montague, Benjamin I. P. Rubinstein

llms multimodal benchmarks safety

arXiv AI

Jun 8

MotionEnhancer: Leveraging Video Diffusion for Motion-Enhanced Vision-Language Models

arXiv:2606. 06853v1 Announce Type: cross Abstract: The new era has witnessed a remarkable capability to extend Vision-Language Models (VLMs) for tackling tasks of video understanding.

By Yifan Xu, Chao Zhang, Ruifei Ma, Fei Gao, Zhifei Yang, Jiaxing Qi, Zhipeng Chen

llms agents diffusion multimodal benchmarks safety

arXiv AI

Jun 8

InvEvolve: Evolving White-Box Inventory Policies via Large Language Models with Performance Guarantees

arXiv:2605. 00369v4 Announce Type: replace-cross Abstract: We study how large language models can be used to generate inventory policies in online settings with non-stationary demand.

By Chenyu Huang, Jianghao Lin, Zhengyang Tang, Bo Jiang, Ruoqing Jiang, Benyou Wang, Lai Wei

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 8

RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs

arXiv:2604. 17948v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across various cybersecurity tasks, including vulnerability classification, detection, and patching.

By Parteek Jamwal, Minghao Shao, Boyuan Chen, Achyuta Muthuvelan, Asini Subanya, Boubacar Ballo, Kashish Satija, Mariam Shafey, Mohamed Mahmoud, Moncif Dahaji Bouffi, Pasindu Wickramasinghe, Siyona Goel, Yaakulya Sabbani, Hakim Hacid, Mthandazo Ndhlovu, Eleanna Kafeza, Sanjay Rawat, Muhammad Shafique

llms rag agents safety

arXiv AI

Jun 8

An Abstract Architecture for Explainable Autonomy in Hazardous Environments

arXiv:2606. 07211v1 Announce Type: cross Abstract: Autonomous robotic systems are being proposed for use in hazardous environments, often to reduce the risks to human workers.

By Matt Luckcuck, Hazel M Taylor, Marie Farrell

agents robotics safety

arXiv Machine Learning

Jun 8

Bias in Filter Feature Selection Evaluation: A Meta-Analysis of Datasets, Baselines, and Experimental Design Choices

arXiv:2606. 07068v1 Announce Type: new Abstract: Background: Since 1990 many feature selection methods have been proposed across heterogeneous applications.

By Malick Ebiele, Malika Bendechache, Rob Brennan

safety

arXiv Machine Learning

Jun 8

Network Recovery from Cascade Data: A Debiased Jacobian-Based Machine Learning Approach

arXiv:2606. 07483v1 Announce Type: new Abstract: Many important outcomes unfold as dynamic cascades, including product adoption, disease spread, financial distress, and information diffusion.

By Lei Huang

diffusion safety

arXiv Machine Learning

Jun 8

The Utility and Complexity of in- and out-of-Distribution Machine Unlearning

arXiv:2412. 09119v3 Announce Type: replace Abstract: Machine unlearning, the process of selectively removing data from trained models, is increasingly crucial for addressing privacy concerns and knowledge gaps post-deployment.

By Youssef Allouah, Joshua Kazdan, Rachid Guerraoui, Sanmi Koyejo

safety

arXiv Machine Learning

Jun 8

Model Recycling Framework for Multi-Source Data-Free Supervised Transfer Learning

arXiv:2508. 02039v2 Announce Type: replace Abstract: Increasing concerns for data privacy and other difficulties associated with retrieving source data for model training have created the need for source-free transfer learning, in which one only has access to pre-trained models instead of data from the original source domains.

By Sijia Wang, Ricardo Henao

fine-tuning safety

arXiv Machine Learning

Jun 8

Adaptive Pluralistic Alignment: A pipeline for dynamic artificial democracy

arXiv:2605. 01642v2 Announce Type: replace Abstract: Prevailing alignment methods target a fixed set of preferences and therefore risk forcing value lock-in as societal norms evolve over time.

By Rachel Freedman

safety

arXiv Machine Learning

Jun 8

GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratios

arXiv:2606. 06967v1 Announce Type: new Abstract: Generative policies provide expressive and multimodal action distributions, making them attractive for reinforcement learning (RL) in complex continuous-control tasks.

By Ke Hu, Shutong Ding, Panxin Tao, Jingya Wang, Ye Shi

reinforcement-learning robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jun 8

Evolving Features vs Evolving Entire Trees with GP for Interpretable Survival Analysis

arXiv:2605. 30119v2 Announce Type: replace-cross Abstract: Survival analysis concerns the task of predicting the time until an event occurs.

By Thalea Schlender, Peter A. N. Bosman, Tanja Alderliesten

safety

Get the daily briefing