AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,474 stories · RSS feed

arXiv Machine Learning

Jun 30

In-Vehicle Digital Twin-Based Collision Warning Framework with Sybil Attack Detection

arXiv:2606. 28625v1 Announce Type: cross Abstract: Connected Vehicles (CVs) rely extensively on communication technologies to enable data-driven predictive analyses for enhancing performance and safety.

By Mohammad Imtiaz Hasan, Abyad Enan, Jean Michel Tine, Araf Rahman, M Sabbir Salek, Mashrur Chowdhury

safety

arXiv Machine Learning

Jun 30

The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives

arXiv:2510. 06096v3 Announce Type: replace Abstract: The objectives that Large Language Models (LLMs) implicitly optimize remain dangerously opaque, making trustworthy alignment and auditing a grand challenge.

By Matthieu Bou, Nyal Patel, Arjun Jagota, Satyapriya Krishna, Sonali Parbhoo

llms reinforcement-learning safety

arXiv Machine Learning

Jun 30

A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions

arXiv:2606. 28943v1 Announce Type: cross Abstract: Learning to bid in repeated multi-unit auctions with bandit feedback poses a fundamental challenge.

By Junhan Li, Yuxin Zhang, Haoran Wang, Minghao Chen

reinforcement-learning benchmarks safety

arXiv AI

Jun 30

Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict

arXiv:2601. 03546v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to simulate decision-making tasks involving personal data sharing, where privacy concerns and prosocial motivations can push choices in opposite directions.

By Guanyu Chen, Chenxiao Yu, Xiyang Hu

llms safety

arXiv AI

Jun 30

The Undecidability of Artificial General Intelligence (AGI) Alignment

arXiv:2606. 28639v1 Announce Type: cross Abstract: This article establishes the foundational mathematical limits of Artificial General Intelligence (AGI) safety, proving that the core barrier is not the impossibility of an aligned state, but its structural unverifiability.

By Jose Pascual Gumbau Mezquita

safety

arXiv AI

Jun 30

Semantic Prompting: Agentic Incremental Narrative Refinement through Spatial Semantic Interaction

arXiv:2604. 19971v2 Announce Type: replace-cross Abstract: Interactive spatial layouts empower users to synthesize information and organize findings for sensemaking.

By Xuxin Tang, Ibrahim Tahmid, Eric Krokos, Kirsten Whitley, Xuan Wang, Chris North

llms agents safety

arXiv AI

Jun 30

Robust Multi-Agent LLMs under Byzantine Faults

arXiv:2605. 09076v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability.

By Haejoon Lee, Vincent-Daniel Yun, Dimitra Panagou, Sai Praneeth Karimireddy

llms agents robotics benchmarks safety

arXiv AI

Jun 30

When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate

arXiv:2512. 03578v3 Announce Type: replace-cross Abstract: Time series extrinsic regression (TSER) refers to the task of predicting a continuous target variable from an input time series.

By Florent Forest, Amaury Wei, Olga Fink

benchmarks safety

arXiv AI

Jun 30

Mechanistically Eliciting Latent Behaviors in Language Models

arXiv:2606. 29604v1 Announce Type: cross Abstract: We aim to discover diverse, generalizable perturbations of LLM internals that can surface hidden behavioral modes.

By Andrew Mack, Nina Panickssery, Alexander Matt Turner

llms safety

arXiv AI

Jun 30

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).

By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao

llms fine-tuning benchmarks safety

arXiv AI

Jun 30

VISTA-DZ: Visual Semantic Trajectory Adaptation for Personalized Dilemma Zone Prediction

arXiv:2606. 29548v1 Announce Type: cross Abstract: Driver decision making in the dilemma zone at signalized intersections is safety critical, as vehicles approaching a yellow signal must decide whether to stop or proceed within limited time and distance margins.

By Chuheng Wei, Ziye Qin, Ziran Wang, Guoyuan Wu

llms rag multimodal safety

arXiv AI

Jun 30

How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models

arXiv:2604. 04385v5 Announce Type: replace-cross Abstract: We localize the policy routing mechanism in alignment-trained language models.

By Gregory N. Frank

llms benchmarks safety

arXiv AI

Jun 30

Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation

arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.

By Gias Uddin, Sanjeepan Sivapiran

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 30

Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense

arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.

By Subhadip Mitra

llms benchmarks safety

arXiv AI

Jun 30

HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning

arXiv:2606. 29126v1 Announce Type: new Abstract: Cooperative multi-agent reinforcement learning (MARL) often relies on communication to mitigate partial observability, yet most existing protocols treat messages as flat dense vectors detached from the structure of the observations they summarize.

By Runze Zhao, Dongruo Zhou, Sumit Kumar Jha, Nathaniel D. Bastian, Ankit Shah

agents reinforcement-learning safety

arXiv AI

Jun 30

Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

arXiv:2606. 29014v1 Announce Type: new Abstract: Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks.

By Dianwei Chen (Terry), Yuan-Zheng Lei (Terry), Zifan Zhang (Terry), Yuchen Liu (Terry), Xianfeng (Terry), Yang

llms agents nlp fine-tuning benchmarks safety

arXiv AI

Jun 30

LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

arXiv:2606. 29399v1 Announce Type: new Abstract: Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters.

By Mingyu Jeon, Bokyeong Kim, Suwan Cho, Jae Young Suh, Yonggyun Yu

llms agents multimodal benchmarks safety

arXiv AI

Jun 30

BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis

arXiv:2606. 29102v1 Announce Type: cross Abstract: Jointly learning to segment and classify medical images demands cross-task synergy, yet encoder-sharing architectures limit decoder reconstruction to task-private representations, permanently discarding the boundary cues and semantic priors each branch could supply to the other.

By Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed, Sk Imran Hossain, Engelbert Mephu Nguifo

computer-vision benchmarks safety

arXiv AI

Jun 30

A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment

arXiv:2606. 29273v1 Announce Type: cross Abstract: Emotion recognition of song lyrics is a challenging task since lyrics may not necessarily align with the overall emotion of a song.

By Rashini Liyanarachchi, Frank Tran, Md Mahmudul Hasan, Aditya Joshi, Erik Meijering

llms safety

arXiv AI

Jun 30

LLM Semantic Signaling Game and Mechanism Design: Systematic Blindness, Awareness Shaping, and Mindset Dynamics

arXiv:2606. 29113v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly mediate strategic interactions through natural language, making semantic control a critical element of communication and deception.

By Quanyan Zhu

llms agents safety

Get the daily briefing