AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 4

EmoDistill: Offline Emotion Skill Distillation for Language Model Agents in Adversarial Negotiation

EmoDistill is an offline framework that distills emotional negotiation skills from large language model interactions into smaller agents. It separates emotion selection, handled by an Implicit Q‑Learning selector, from emotion‑conditioned expression, learned by a LoRA‑adapted 7B policy via supervised fine‑tuning and judge policy optimization. Experiments across four negotiation domains show that the full EmoDistill policy outperforms vanilla and IQL‑only baselines, while removing the explicit emotion channel markedly reduces negotiation utility and reveals partial, domain‑dependent transfer to unseen counterparties.

By Yunbo Long, Haolang Zhao, Lukas Beckenbauer, Liming Xu, Alexandra Brintrup
arXiv AI
Sep 4

DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents

DuplexSpeechBench-IFEval (DSB-IFEval) is a new benchmark that evaluates how full‑duplex voice agents follow implicit instructions during real‑time spoken interaction. It contains 1,038 test cases across eight assistant roles and tests five conditioning protocols, measuring floor management with an Instruction Adherence Score (IAS) and persona consistency with a Persona Adherence Score (PAS). Experiments on six speech systems reveal architecture‑dependent trade‑offs, showing that some models are more sensitive to explicit versus persona‑only instructions and that even when following conflicting directives, they struggle to override them under safety conflict.

By Puneet Mathur, Dinesh Manocha
arXiv AI
Sep 4

Fresh Memory, Stale Plans: Dependency-Scoped Validation for Distributed LLM-Agent Memory

The paper introduces PlanFence, a dependency-scoped action‑validation protocol for distributed large language model (LLM) agent teams. PlanFence requires plans to cite the exact public records they rely on, and executors validate only those records that could affect the pending action, replanning or blocking if validation is incomplete. In 30 controlled live workflows, a freshness‑only executor always acted on obsolete plans, whereas PlanFence completed all tasks without invalid actions, demonstrating controlled safety and system‑cost benefits.

By Evan Chen, Shiqiang Wang, Christopher G. Brinton
arXiv AI
Sep 4

Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty

The paper introduces Provenance Density, an interface that visualizes the density of verified claims within a text to counter the Fluency Trap—where users mistake fluent AI-generated hallucinations for truth. In a study with 81 participants, the interface significantly improved users’ ability to distinguish true from fabricated content, while no signal led to no discernment. A technical audit of 200 samples revealed that retrieval density alone is insufficient, and that the Consistency Veto provides most of the discriminative power for dynamic queries.

By Qing Zhang, Yifei Huang, Juyoung Lee, Thad Starner, Jun Rekimoto
arXiv AI
Sep 4

Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment

The paper argues that in offline preference optimization for reasoning models, applying gradients uniformly to all chosen–rejected pairs is inefficient and can be harmful. It introduces the concept of gradient utility, showing that a pair’s contribution depends on both informativeness and stability, and that high-gradient samples often lie in high‑curvature regions, causing noisy updates. To address this, the authors propose SAGE (Stability‑Aware Gradient Efficiency), which maintains difficulty‑stratified candidate pools and selects only high‑utility pairs for backpropagation, resulting in smoother optimization and better performance on mathematical reasoning benchmarks.

By Hui Wu, Hengyi Cai, Jinman Zhao, Xinran Chen, Ziheng Li, Zhejun Zhao, Shuaiqiang Wang, Yuchen Li, Dawei Yin
arXiv Machine Learning
Sep 4

Portable Causal Fairness Across Synthetic Data Generator Families

The paper demonstrates that causal fairness mechanisms can be applied across a wide range of synthetic data generators, including marginal‑based, GAN, and diffusion models, each with differentially private variants. By porting three fairness definitions to nine generators and testing them on Adult and COMPAS datasets, the authors show that the causal diffusion backbone consistently produces the fairest data releases while maintaining high fidelity. The fairness cuts have minimal impact on data quality, costing downstream classifiers only about $0.07$ to $0.15$ AUC on average, and adding privacy guarantees does not reduce fairness.

By Steven Golob, Sikha Pentyala, Martine De Cock
arXiv Machine Learning
Sep 4

Hadronic Mono-Z Dark Matter Sensitivity with Flow Matching on CMS Open Data

The paper projects the sensitivity of a hadronic mono‑Z dark‑matter search using CMS Run 2015D HTMHT open data (2.256 fb⁻¹). A conditional flow‑matching normalizing flow models backgrounds, with careful handling of missing features and a sentinel imputation strategy. The baseline analysis yields expected significances of 2.89σ, 7.62σ, and 7.41σ for three benchmark models, and an ablation study shows that extra‑jet kinematics contribute 53–71% of the discriminating power.

By Hitesh Rasineni (VIT-AP University, Amaravati, India), Bhavishya Chebrolu (Mohan Babu University, Tirupati, India)
arXiv Machine Learning
Sep 4

Privacy Leakage in Federated Learning: Gradient-Based Client Identity Inference and Defenses for Inertial Sensing in Vehicular Edge Networks

The paper investigates how federated learning (FL) updates in vehicular edge networks can reveal client identities through gradient-based attacks on inertial sensor data, using the UCI Human Activity Recognition benchmark as a proxy. Experiments show that an honest-but-curious server can identify clients with near-perfect accuracy from unprotected updates. The authors evaluate lightweight defenses—clipping followed by Gaussian noise and ensemble FL—to mitigate this privacy risk while preserving model utility, reporting differential‑privacy budgets and empirical results across multiple attack classifiers and data partitions.

By Ali Akarma (Islamic University of Madinah, Madinah, Saudi Arabia, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia), Toqeer Ali Syed (Islamic University of Madinah, Madinah, Saudi Arabia), Muhammad Khan (University of the West of England, Bristol, U.K), Qurat-ul-ain Mastoi (University of the West of England, Bristol, U.K), Adeel Ahmad (Islamic University of Madinah, Madinah, Saudi Arabia)
arXiv Machine Learning
Sep 4

ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models

The paper introduces Adaptive Local Relational Alignment (ALRA), a logit‑based knowledge distillation method for autoregressive language models that combines student‑generated token proposals with teacher guidance at each prediction position. ALRA dynamically selects the number of candidate tokens based on the teacher’s probability spread, uses Adaptive Local Divergence to match both mass and relative token distributions, and applies Student‑Weighted Pairwise Relational Alignment to focus on high‑probability token pairs. Experiments on The Pile show that 200M‑ and 500M‑parameter students trained with ALRA outperform the best baseline by roughly 1 percentage point and surpass pre‑training without distillation by over 2 percentage points on nine zero‑shot benchmarks.

By Quang Hoang Trung, Quang Huu Hieu, Nguyen Van Hoang Phuc, Vo Nguyen Le Duy
arXiv Machine Learning
Sep 4

LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models

The paper introduces PromptGFM, a Graph Foundation Model designed for text‑attributed graphs (TAGs). It integrates Large Language Models (LLMs) and Graph Neural Networks (GNNs) through a Graph Understanding Module that prompts LLMs to emulate GNN workflows, and a Graph Inference Module that creates a language‑based graph vocabulary for better alignment and scalability. Experiments show PromptGFM outperforms existing methods and transfers effectively across various graphs and tasks.

By Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, Imran Razzak, Yongfeng Zhang
arXiv Computation and Language
Sep 4

RL-ADA: A World-Feedback Framework for Adversarially Robust Enterprise Dialogue Agents

RL-ADA introduces a co‑evolutionary training framework that replaces costly human annotations with world‑feedback rewards derived from interaction outcomes. In this system, a large Customer Support Agent and an Adversarial Customer Agent train together, guided by an automated judge that rewards successful resolution and realistic intent‑concealing utterances, respectively. Applied to a banking support proof of concept, the method eliminates routing errors and doubles the end‑to‑end PASS rate over five cycles, while also revealing a new adversarial strategy called Contextual Camouflage.

By Ram Narayanan, Harshit Rajgarhia, Abhishek Mukherji
arXiv Computation and Language
Sep 4

Large Language Models in Resolving Contextual Knowledge Conflicts

The paper introduces a taxonomy of six types of contextual knowledge conflicts—factual, inferential, temporal, granularity, perspective, and ambiguity—and presents the ContextConflict dataset with 5,781 samples covering reasoning and summarization tasks. Experiments on nine large language models reveal that current models struggle to resolve these conflicts, exhibit a bias toward earlier evidence, and show latent awareness of conflicts in their internal representations. The authors propose a training‑free, label‑free steering method that adjusts activations to better incorporate evidence, consistently improving reasoning accuracy and producing higher‑quality, balanced summaries on the dataset.

By Xinye Yang, Zhenyang Liu, Ruisi Li, Yuanyuan Lei
arXiv Computation and Language
Sep 4

No country for old linguists: LLM-brain alignment underdetermines neural computation

Nastase et al. (2026) argue that large language models (LLMs) can shed light on language processing because both use distributed, context‑sensitive representations shaped by statistical learning, and they advocate for LLM‑brain alignment research. They reject simple cortical “boxology” but claim that representational alignment can constrain mechanistic hypotheses, though it does not itself identify a mechanism. The author critiques this position, pointing out logical, causal, and computational underdetermination and the tension between the authors’ methodological caveats and their conclusion that LLMs could serve as fully mechanistic models of language.

By Elliot Murphy
arXiv Computation and Language
Sep 4

How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models

The paper investigates how six naturalistic and synthetic input perturbations affect decoder‑only language models at three levels: output behavior, hidden‑state geometry, and attention‑head function. Using GPT‑2 and Qwen2.5 checkpoints, the authors analyze layerwise geometry with centered kernel alignment and intrinsic dimension, and examine attention‑head responses in GPT‑2. They find that perturbation types produce distinct metric profiles that are not fully captured by output measures and vary across checkpoints, highlighting the need for multi‑level evaluation of robustness.

By Dun Li Chan, Emily Liu, Niyathi Allu, Christian Hoang
arXiv Computation and Language
Sep 4

A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities

The paper investigates how large language models (LLMs) represent and retrieve singular and plural entities for coreference resolution. Using mechanistic interpretability and attention pattern analysis, the authors identify specific attention heads that encode coreference information, detect plural entities, and guide pronoun prediction. The study also shows that LLMs prefer plural pronouns for ontologically similar entities linked by "and", mirroring human preferences.

By Anh Danh, Rick Nouwen, Massimo Poesio
arXiv Computation and Language
Sep 4

Causal-Counterfactual RAG: The Integration of Causal-Counterfactual Reasoning into RAG

The paper introduces Causal-Counterfactual RAG, a new framework that augments Retrieval-Augmented Generation with explicit causal graphs and counterfactual reasoning. By incorporating cause‑effect relationships into retrieval and evaluating both direct causal evidence and counterfactual scenarios, the approach aims to produce more robust, accurate, and interpretable answers. This method seeks to maintain contextual coherence, reduce hallucinations, and improve reasoning fidelity compared to traditional RAG systems.

By Harshad Khadilkar, Abhay Gupta
arXiv Computation and Language
Sep 4

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.

By Michael Li, Nishant Subramani
arXiv Computer Vision
Sep 4

Preprocessing Failure and Adversarial Detection in Depthwise-Separable Edge Vision Systems

The paper evaluates six preprocessing defenses against adversarial attacks on depthwise‑separable CNNs, the dominant architecture in edge vision systems, and finds that these defenses consistently fail to recover clean predictions for such models, whereas a residual architecture shows partial recovery. The study reveals that the same preprocessing steps that break clean predictions leave adversarial predictions largely intact, creating a measurable asymmetry that can be exploited for detection without retraining or architectural changes. It also demonstrates that common image quality metrics do not reliably indicate defense effectiveness, highlighting a methodological gap in current evaluation practices.

By Jannatul Masruk Mukta, Rifa Sanjida, Adrita Rahman Tory, Md. Saifur Rahman, Khondokar Fida Hasan
arXiv Computer Vision
Sep 4

SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Models

SafeRI proposes an on-demand safety alignment approach for large vision-language models, contrasting with existing always-on methods that globally modify model behavior. The framework uses a lightweight recognizer to evaluate token-level safety during autoregressive generation, gating a LoRA module that only activates when unsafe content is detected. By training the LoRA on unsafe prefixes and safe continuations, SafeRI redirects unsafe generations back to safe responses without perturbing the model’s original reasoning path.

By Caoyuan Ma, Tian Gu, Wenpu Liu, Weichu Xie, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Zheng Wang, Yinqiang Zheng
arXiv Computer Vision
Sep 4

TokenMatch: 3D Mesh Correspondence Transformer with Curvature-Guided Tokenisation

TokenMatch is a transformer-based model that estimates 3D shape correspondences by adaptively tokenising meshes into curvature-guided patches. Trained only on the BeCoS partial-to-partial dataset, it generalises to full-shape matching without retraining, using self‑ and cross‑attention to learn patch‑ and point‑level relations. Evaluated on CP2P, PSMAL, BeCoS, FAUST, SCAPE, and SHREC'19, TokenMatch consistently outperforms existing methods in mean geodesic error and intersection‑over‑union while achieving sub‑second inference speeds.

By Adeela Islam, Zorah L\"ahner, Vittorio Murino, Vladislav Golyanik