AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,825 stories · RSS feed

arXiv AI

Jun 6

Output Type Before Quality: A Standards-Derived XAI Admissibility Rubric for Autonomous-Driving Safety

arXiv:2606. 05461v1 Announce Type: new Abstract: Safety standards for ML-based autonomous driving specify the kind of evidence an assurance case must contain (directed cause-and-effect chains, quantified interventional effects, named root-cause variables), yet the XAI literature is organised by output type and technique family (saliency maps, feature attribution, counterfactuals, causal graphs, language traces).

By Abhinaw Priyadershi, Mandar Pitale, Jelena Frtunikj, Maria Spence

agents robotics safety

arXiv AI

Jun 6

DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments

arXiv:2606. 06217v1 Announce Type: cross Abstract: When a disaster unfolds, responders must answer not only what is happening, but also why it is happening, what will happen next, and what to do now, often from noisy low-altitude UAV views and under tight on-site compute constraints.

By Tan Zhang, Quanyou Li, Lu Zhang, Jun Liu, Xiaofeng Zhu, Ping Hu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 6

Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

arXiv:2606. 05688v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale foundation models efficiently by activating only a subset of experts for each token, but their large number of expert parameters still makes quantization essential for practical deployment.

By Hancheol Park, Geonho Lee, Tairen Piao, Tae-Ho Kim

efficiency safety

arXiv AI

Jun 6

Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning

arXiv:2503. 01734v3 Announce Type: replace-cross Abstract: Attacks on machine learning models have been extensively studied through stateless optimization.

By Kyle Domico, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Eric Pauley, Josiah Hanna, Patrick McDaniel

agents computer-vision reinforcement-learning benchmarks safety

arXiv AI

Jun 6

CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications

arXiv:2512. 15231v3 Announce Type: replace Abstract: The automated and intelligent processing of massive remote sensing (RS) datasets is critical in Earth observation (EO).

By Zhengchao Chen, Haoran Wang, Jing Yao, Jianshe Zhang, Pedram Ghamisi, Jun Zhou, Peter M. Atkinson, Bing Zhang

llms rag agents benchmarks safety

arXiv AI

Jun 6

Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration

arXiv:2606. 05396v1 Announce Type: cross Abstract: Producing a labeled vulnerable code at scale is a recurring obstacle for learning-based vulnerability detection: mined corpora carry substantial label noise, and existing LLM-based augmentation propagates these inaccuracies because it transforms vulnerable seeds rather than synthesising vulnerabilities from a specification.

By Cristina Carleo, Pietro Liguori, Naghmeh Ivaki, Domenico Cotroneo

llms reinforcement-learning safety data-engineering

arXiv AI

Jun 6

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.

By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan

llms agents safety

arXiv AI

Jun 6

Generating Graph-Like Logical Rules for Knowledge Graph Reasoning via Diffusion Models

arXiv:2605. 30747v2 Announce Type: replace Abstract: Logical rules constitute a cornerstone of knowledge graph (KG) reasoning, valued for their interpretability and ability to model relational patterns.

By Haoxiang Cheng, Yunfei Wang, Chao Chen, Kewei Cheng, Zhipeng Lin, Haoxuan Li, Changjun Fan, Shixuan Liu

diffusion reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 6

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

arXiv:2606. 05644v1 Announce Type: new Abstract: When retrieved evidence contradicts parametric memory, language models frequently ignore context and default to memorized priors -- a failure that undermines the core purpose of retrieval augmentation.

By Zhe Yu, Wenpeng Xing, Tiancheng Zhao, Mohan Li, Changting Lin, Meng Han

llms rag benchmarks safety

arXiv AI

Jun 6

Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents

arXiv:2604. 00555v5 Announce Type: replace Abstract: Enterprise adoption of Large Language Models (LLMs) is constrained by hallucination, domain drift, and the inability to enforce regulatory compliance at the reasoning level.

By Thanh Luong Tuan, Abhijit Sanyal

llms agents safety data-engineering

arXiv AI

Jun 6

MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

arXiv:2605. 28579v2 Announce Type: replace Abstract: Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design.

By Xiaoyu Dong, Zhi Li, Xiao-Ming Wu

llms multimodal benchmarks safety

arXiv AI

Jun 6

Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO

arXiv:2606. 05174v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown strong promise in healthcare applications.

By Arash Ahmadi, Parisa Masnadi, Sarah Sharif, Charles Nicholson, David Ebert, Mike Banad

llms nlp reinforcement-learning safety

arXiv AI

Jun 6

Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models

arXiv:2606. 05702v1 Announce Type: new Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly enhanced their ability to interpret complex visual semantics, yet their capacity for chronological reasoning remains under-explored.

By Haoyu Zhou, Qing Qing, Caichong Li, Qixin Zhang, Yongcheng Jing, Ziqi Xu, Juncheng Hu, Xikun Zhang, Renqiang Luo

llms multimodal benchmarks safety

arXiv AI

Jun 6

UNIVID: Unified Vision-Language Model for Video Moderation

arXiv:2606. 05748v1 Announce Type: cross Abstract: Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement.

By Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao

llms multimodal safety

arXiv AI

Jun 6

An Infectious Disease Spread Simulation Based on Large Language Model Decision Making

arXiv:2606. 06360v1 Announce Type: new Abstract: Modelling individual decision-making during infectious disease outbreaks is crucial for understanding behavioural dynamics and informing effective public health interventions.

By Yonchanok Khaokaew, Ruochen Kong, Andreas Zufle, Hao Xue, Taylor Anderson, Chandini Raina MacIntyre, Matthew Scotch, Flora D. Salim, David J Heslop

llms agents safety

arXiv AI

Jun 6

Synapse: Federated Tool Routing via Typed Compendium Artifacts

arXiv:2602. 00911v2 Announce Type: replace Abstract: The unit of collaboration in federated learning determines what guarantees are even expressible.

By Abhijit Chakraborty, Yash Shah, Vivek Gupta

llms safety

arXiv AI

Jun 6

LLM Self-Recognition: Steering and Retrieving Activation Signatures

arXiv:2606. 06315v1 Announce Type: new Abstract: Recent advances in interpretability suggest that large language models (LLMs) implicitly encode signals in their generated text that enable self-recognition of their outputs.

By Thibaud Ardoin, Jonas Sch\"afer, Gerhard Wunder

llms rag safety

arXiv AI

Jun 6

Whose Alignment? Comparing LLM Process Alignment Across Diverse Organizational Decision Contexts

arXiv:2605. 25256v2 Announce Type: replace Abstract: Steerable pluralism requires a model to faithfully represent one specified perspective.

By Niklas Weller, Emilio Barkett

llms benchmarks safety

arXiv AI

Jun 6

Insurance of Agentic AI

arXiv:2606. 05449v1 Announce Type: new Abstract: Agentic artificial intelligence (AI) systems are transforming the risk landscape by extending beyond information generation to autonomous planning, tool invocation, decision execution, and persistent modification of digital and physical environments.

By Quanyan Zhu

agents safety

arXiv AI

Jun 6

When AI Says It Feels

arXiv:2606. 05734v1 Announce Type: new Abstract: Large language models (LLMs) are generally constrained from expressing feelings through human-preference alignment in post-training processes.

By Shin-nosuke Ishikawa, Seiya Ikeda, Hirotsugu Ohba

llms reinforcement-learning safety

Get the daily briefing