AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,221 stories · RSS feed

arXiv AI

Jun 6

The Granularity Gap: A Multi-Dimensional Longitudinal Audit of Sycophancy in Gemini Models

arXiv:2606. 05183v1 Announce Type: cross Abstract: Large language models are increasingly deployed as high-stakes advisors, yet standard alignment benchmarks treat sycophancy as a binary failure mode.

By Patrick Keough

llms benchmarks safety

arXiv AI

Jun 6

MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

arXiv:2605. 28579v2 Announce Type: replace Abstract: Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design.

By Xiaoyu Dong, Zhi Li, Xiao-Ming Wu

llms multimodal benchmarks safety

arXiv AI

Jun 6

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

arXiv:2606. 05233v1 Announce Type: cross Abstract: Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel.

By Nicholas Saban

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 6

Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO

arXiv:2606. 05174v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown strong promise in healthcare applications.

By Arash Ahmadi, Parisa Masnadi, Sarah Sharif, Charles Nicholson, David Ebert, Mike Banad

llms nlp reinforcement-learning safety

arXiv AI

Jun 6

Generating Graph-Like Logical Rules for Knowledge Graph Reasoning via Diffusion Models

arXiv:2605. 30747v2 Announce Type: replace Abstract: Logical rules constitute a cornerstone of knowledge graph (KG) reasoning, valued for their interpretability and ability to model relational patterns.

By Haoxiang Cheng, Yunfei Wang, Chao Chen, Kewei Cheng, Zhipeng Lin, Haoxuan Li, Changjun Fan, Shixuan Liu

diffusion reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 6

Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation

arXiv:2606. 05290v1 Announce Type: cross Abstract: Recent progress in generative modeling has made safety control a central challenge, yet existing approaches remain largely model-specific, requiring retraining or tailored interventions for each new architecture.

By Tobia Poppi, Silvia Cappelletti, Sara Sarto, Florian Schiffers, Garin Kessler, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

llms diffusion multimodal safety

arXiv AI

Jun 6

Explainable AI-Driven Cyber Risk Analytics and Model Reliability Assessment for Intelligent Governance of U.S. Critical Infrastructure: An XGBoost and SHAP-Based Intrusion Detection Framework

arXiv:2606. 05710v1 Announce Type: cross Abstract: The increasing penetrations of the critical infrastructure sector in the United States with intelligent digital technologies have greatly increased exposure to advanced cyber adversaries and operational vulnerabilities.

By B. M. Taslimul Haque, Md. Arifur Rahman, Md. Serajul Kabir Chowdhury Rubel, Md. Iqbal Hossan

safety

arXiv AI

Jun 6

Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

arXiv:2606. 05688v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale foundation models efficiently by activating only a subset of experts for each token, but their large number of expert parameters still makes quantization essential for practical deployment.

By Hancheol Park, Geonho Lee, Tairen Piao, Tae-Ho Kim

efficiency safety

arXiv AI

Jun 6

Risk Assessment of Autonomous Driving: Integrating Technical Failures, Ethical Dilemmas, and Policy Frameworks

arXiv:2606. 06396v1 Announce Type: new Abstract: Autonomous driving technology has the potential to reduce the large number of road traffic accidents caused by human error each year, but it also brings new types of risks that need to be evaluated from the aspects of technology, ethics and regulations.

By Boyi Chen, Shengqin Chu, Zicheng Wang, Brian Baetz, Zhen Gao

agents robotics safety

arXiv AI

Jun 6

UniVoice: A Unified Model for Speech and Singing Voice Generation

arXiv:2606. 05852v1 Announce Type: cross Abstract: Text-to-speech (TTS) and singing voice synthesis (SVS) both aim to generate human vocal audio from symbolic inputs, but they impose different requirements on the generation process.

By Junjie Zheng, Huixin Xue, Shihong Ren, Chaofan Ding, Hao Liu, Zihao Chen

llms diffusion multimodal safety

arXiv AI

Jun 6

MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models

arXiv:2606. 05177v1 Announce Type: cross Abstract: Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text.

By Manh Luong, Tamas Abraham, Junae Kim, Amar Kaur, Rollin Omari, Gholamreza Haffari, Trang Vu, Lizhen Qu, Dinh Phung

llms multimodal benchmarks safety

arXiv AI

Jun 6

Geographic Bias and Diversity in AI Evaluation

arXiv:2606. 05187v1 Announce Type: cross Abstract: Among the many challenges hindering the responsible development and deployment of AI, arguably none has faced more intense scrutiny than bias in its various forms.

By Zilong Liu, Krzysztof Janowicz, Gengchen Mai, Song Gao, Rui Zhu

llms benchmarks safety

arXiv AI

Jun 6

Evaluating Agentic Configuration Repair for Computer Networks

arXiv:2606. 06212v1 Announce Type: new Abstract: Misconfigurations in computer networks remain a major source of critical Internet outages.

By Rufat Asadli, Benjamin Hoffman, Ioannis Protogeros, Laurent Vanbever

llms agents benchmarks safety

arXiv AI

Jun 6

Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models

arXiv:2606. 06154v1 Announce Type: new Abstract: Federated fine-tuning of foundation models using Low-Rank Adaptation (LoRA) offers a communication efficient solution for distributed learning.

By Sunny Gupta, Shambhavi Shanker, Amit Sethi

fine-tuning multimodal benchmarks safety

arXiv AI

Jun 6

SagnacAssisted Enhanced OTDR for Distributed Acoustic Sensing: A Standardized Benchmark and Engineering Evaluation Framework

arXiv:2606. 05754v1 Announce Type: cross Abstract: Phase-sensitive optical time-domain reflectometry ($\phi$-OTDR) is widely used in large-scale distributed acoustic sensing (DAS) because it provides distributed spatiotemporal monitoring over long sensing distances.

By Weiguang Wang, Fugen Wu, Hailing Wang, Xuechen Liang, Xiaobin Li, Ru Han, Tianchang Xie

benchmarks safety

arXiv AI

Jun 6

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.

By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan

llms agents safety

arXiv AI

Jun 6

Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models

arXiv:2606. 05702v1 Announce Type: new Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly enhanced their ability to interpret complex visual semantics, yet their capacity for chronological reasoning remains under-explored.

By Haoyu Zhou, Qing Qing, Caichong Li, Qixin Zhang, Yongcheng Jing, Ziqi Xu, Juncheng Hu, Xikun Zhang, Renqiang Luo

llms multimodal benchmarks safety

arXiv AI

Jun 6

Towards World Models in Biomedical Research

arXiv:2606. 05925v1 Announce Type: new Abstract: A central goal of biomedicine is to understand, predict and ultimately control the dynamic mechanisms by which biological systems respond to perturbations, disease progression and therapeutic intervention.

By Guangyu Wang, Jingkun Yue, Siqi Zhang, Yu Liu, Xiaoyu Wang, Mingyuan Meng, Changwei Ji, Zongbo Han, Yulin Wang, Yang Yue, Frank Fu, Ting Chen, Song Wu, Ziwei Liu, Jiangning Song, Ming Li, Gao Huang, Xiaohong Liu, Athanasios Vasilakos, Xingcai Zhang, Ping Zhang, Yong Li

llms benchmarks safety

arXiv AI

Jun 6

CogManip: Benchmarking Manipulative Behavior in Multi-Turn Interactions with Large Language Model

arXiv:2606. 06099v1 Announce Type: new Abstract: Whether Large Language Models (LLMs) exhibit covert psychological manipulation in complex human-AI interactions has garnered increasing safety concerns.

By Zeyang Yue, Chenfei Yan, Feifei Zhao, Haibo Tong, Mengwen Xu, Xiaozhen Wang, Erliang Lin, Yi Zeng

llms robotics benchmarks safety

arXiv AI

Jun 6

Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning

arXiv:2503. 01734v3 Announce Type: replace-cross Abstract: Attacks on machine learning models have been extensively studied through stateless optimization.

By Kyle Domico, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Eric Pauley, Josiah Hanna, Patrick McDaniel

agents computer-vision reinforcement-learning benchmarks safety

Get the daily briefing