AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,221 stories · RSS feed

arXiv AI

Jun 6

An Infectious Disease Spread Simulation Based on Large Language Model Decision Making

arXiv:2606. 06360v1 Announce Type: new Abstract: Modelling individual decision-making during infectious disease outbreaks is crucial for understanding behavioural dynamics and informing effective public health interventions.

By Yonchanok Khaokaew, Ruochen Kong, Andreas Zufle, Hao Xue, Taylor Anderson, Chandini Raina MacIntyre, Matthew Scotch, Flora D. Salim, David J Heslop

llms agents safety

arXiv AI

Jun 6

LatentWave: JEPA Pretraining for Wireless Foundation Models

arXiv:2606. 06373v1 Announce Type: cross Abstract: Wireless foundation models have emerged as a promising alternative to building separate models for each wireless task.

By Ahmed Mohamed, Ahmed Aboulfotouh, Hatem Abou-Zeid

rag diffusion safety

arXiv AI

Jun 6

Compositional Boundaries for Density Fusion

arXiv:2606. 05871v1 Announce Type: cross Abstract: Distributed uncertainty-management systems often combine local probabilistic models along aggregation trees chosen by communication, privacy, or scheduling constraints.

By Ratan Bahadur Thapa, Ali Darijani, J\"urgen Beyerer, Steffen Staab

safety

arXiv AI

Jun 6

Evaluating the Utility of Personal Health Records in Personalized Health AI

arXiv:2605. 18937v2 Announce Type: replace Abstract: Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex, potentially hindering insights.

By Rory Sayres, Kejia Chen, Ayush Jain, Matthew Thompson, Jonathan Richina, Xiang Yin, Jimmy Hu, Fan Zhang, Bob Lou, Mike Sanchez, Ines Mezerreg, Meredith Schreier, Hamsa Subramaniam, I-Ching Lee, Yugang Jia, Daniel Mcduff, Yossi Matias, Avinatan Hassidim, Dale Webster, Yun Liu, Jackie Barr, Quang Duong

llms safety

arXiv AI

Jun 6

Beyond Waveform Robustness: Robust Feature-Vocoder Adversarial Attacks on Automatic Speech Recognition

arXiv:2606. 05678v1 Announce Type: cross Abstract: Automatic speech recognition (ASR) systems have become widely used for multilingual speech-to-text transcription.

By Yifan Liao, Zongmin Zhang, Zhen Sun, Yuhui Sun, Xinhu Zheng, Xinlei He

multimodal benchmarks safety

arXiv AI

Jun 6

CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives

arXiv:2504. 10823v4 Announce Type: replace-cross Abstract: Navigating dilemmas involving conflicting values is challenging even for humans in high-stakes domains, let alone for AI, yet prior work has been limited to everyday scenarios.

By Ayoung Lee, Ryan Sungmo Kwon, Peter Railton, Lu Wang

llms benchmarks safety

arXiv AI

Jun 6

From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment

arXiv:2606. 05180v1 Announce Type: cross Abstract: Automated scoring models are increasingly used to assign rubric-based quality ratings to complex language performances, including classroom transcripts, yet they typically provide little insight into why a particular score is produced.

By Ivo Bueno, Babette B\"uhler, Philipp Stark, Tim F\"utterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci

llms fine-tuning safety

arXiv AI

Jun 6

Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing

arXiv:2605. 04733v2 Announce Type: replace Abstract: Text-based role-playing models can imitate character styles, but often fail to capture scene atmosphere and evolving tension, which are crucial for immersive applications such as VR games and interactive narratives.

By Miao Wang, Yuling Shi, Yijiang Li, Yeheng Chen, Xiaodong Gu, Bin Li, Bo Gao, Jun Wang, Zengxin Han, Jingtong Wu, Yaduan Ruan

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 6

UNIVID: Unified Vision-Language Model for Video Moderation

arXiv:2606. 05748v1 Announce Type: cross Abstract: Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement.

By Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao

llms multimodal safety

arXiv AI

Jun 6

Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation

arXiv:2606. 05660v1 Announce Type: cross Abstract: Embodied AI systems are increasingly expected to reason and act over extended horizons in physical environments.

By Dabin Kim, Daemin Park, Sangyub Lee, Jinsik Kim, Yeongtak Oh, Jongho Shin, Sungroh Yoon

agents robotics benchmarks safety

arXiv AI

Jun 6

Generating Graph-Like Logical Rules for Knowledge Graph Reasoning via Diffusion Models

arXiv:2605. 30747v2 Announce Type: replace Abstract: Logical rules constitute a cornerstone of knowledge graph (KG) reasoning, valued for their interpretability and ability to model relational patterns.

By Haoxiang Cheng, Yunfei Wang, Chao Chen, Kewei Cheng, Zhipeng Lin, Haoxuan Li, Changjun Fan, Shixuan Liu

diffusion reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 6

MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

arXiv:2605. 28579v2 Announce Type: replace Abstract: Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design.

By Xiaoyu Dong, Zhi Li, Xiao-Ming Wu

llms multimodal benchmarks safety

arXiv AI

Jun 6

Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning

arXiv:2503. 01734v3 Announce Type: replace-cross Abstract: Attacks on machine learning models have been extensively studied through stateless optimization.

By Kyle Domico, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Eric Pauley, Josiah Hanna, Patrick McDaniel

agents computer-vision reinforcement-learning benchmarks safety

arXiv AI

Jun 6

Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads

arXiv:2606. 05843v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate remarkable proficiency on complex vision-language tasks, the mechanisms by which they extract query-relevant visual features from complex, noisy contexts remain opaque.

By Ruoxi Sun, Quantong Qiu, Juntao Li, Zecheng Tang, Yihang Lou, Min Zhang

llms efficiency multimodal safety

arXiv AI

Jun 6

PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage

arXiv:2606. 05463v1 Announce Type: new Abstract: Patient safety event triage, determining whether a clinical event is reportable under jurisdiction-specific policy, is a high-stakes task typically performed manually by patient safety experts.

By Keqi Han, Ryan Young, Annabel Strauss, Lindsey Hughes, Katharine M. Nesbitt, Nicole Schueler, Che Ngufor, Carl Yang, Yuan Xue, Zhijun Yin

llms agents benchmarks safety

arXiv AI

Jun 6

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

arXiv:2606. 05233v1 Announce Type: cross Abstract: Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel.

By Nicholas Saban

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 6

VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents

arXiv:2606. 05395v1 Announce Type: cross Abstract: Reusable robot skills are becoming the basic units through which embodied agents turn open-ended instructions into long-horizon physical behavior.

By Yunhao Yang, Neel P. Bhatt, Kevin Wang, Samuel Tetteh, Zhangyang Wang, Ufuk Topcu

llms agents robotics fine-tuning safety

arXiv AI

Jun 6

GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection

arXiv:2606. 05566v1 Announce Type: new Abstract: Large Language Models (LLMs) have transformed natural language processing, but they remain vulnerable to Prompt Injection (PI) and Jailbreak (JB) attacks.

By Paulo Ricardo Ferreira Neves, Edson Rodrigues da Cruz Filho, Paulo Henrique Eleuterio Falsetti, Jo\~ao Vitor Pavan, Ian Degaspari, Henrique Vieira Laturrague, Patrick Vieira Laturrague, Guilherme Nielsen Dias, Marccello Wilson Perez Berto, Gustavo Voltani Von Atzingen

llms nlp benchmarks safety

arXiv AI

Jun 6

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

arXiv:2606. 05644v1 Announce Type: new Abstract: When retrieved evidence contradicts parametric memory, language models frequently ignore context and default to memorized priors -- a failure that undermines the core purpose of retrieval augmentation.

By Zhe Yu, Wenpeng Xing, Tiancheng Zhao, Mohan Li, Changting Lin, Meng Han

llms rag benchmarks safety

arXiv AI

Jun 6

Harnessing Structural Context for Entity Alignment Foundation Models

arXiv:2606. 06109v1 Announce Type: cross Abstract: Entity alignment (EA) aims to identify equivalent entities across heterogeneous knowledge graphs (KGs) and is a key component of knowledge fusion and cross-KG reasoning.

By Xingyu Chen, Yuanning Cui, Zequn Sun, Wei Hu

benchmarks safety

Get the daily briefing