AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Sep 3

InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models

InfraPatch is a white‑box, per‑instance framework that generates small grayscale patches to target infrared‑adapted vision‑language models (IR‑VLMs). The method optimizes a single‑channel patch within a 5% local‑area budget, using proxy‑guided placement and task‑adaptive objectives to induce desired behaviors in image classification, captioning, and binary visual question answering. Across ten IR‑VLM variants tested on synthetic infrared images, InfraPatch achieves targeted attack success rates ranging from 86% to 100%, revealing significant vulnerability differences among architectures and tasks.

By Chengyin Hu, Dingyi Lu, Jiaju Han, Xiang Chen, Weiwen Shi, Jiahuan Long, Yiwei Wei, Jiujiang Guo
arXiv AI
Sep 3

CrashDiffuser: VLM-Guided Collision Intent Reasoning for Fine-Grained Safety-Critical Traffic Scenario Generation

CrashDiffuser is a closed-loop VLM‑guided diffusion framework designed for fine‑grained safety‑critical traffic scenario generation. It separates semantic collision reasoning from trajectory synthesis via a hierarchical collision‑intent interface that specifies target contact regions (head, rear, or side). The system uses a vision‑language model to extract scene context and predict structured action tuples, which condition a diffusion model to produce executable adversarial trajectories, achieving high target‑collision and contact‑region control rates on WOMD‑derived scenarios.

By Shucheng Zhang, Yuang Zhang, Bingzhang Wang, Muhammad Monjurul Karim, Kehua Chen, Yinhai Wang
arXiv AI
Sep 3

Before the Script, Set the Stage: How Worldview Simulation Amplifies Psychologically Grounded Persuasion in Multi-Turn Jailbreaking

The paper introduces BLUEPRINT, a safety‑evaluation framework that separates a factorized social‑influence strategy space from WORLDVIEWSIM, a cross‑turn situational context module. Using Monte Carlo Tree Search, it optimizes turn‑level combinations of 18 theory‑grounded influence factors across a four‑turn trajectory, achieving near‑ceiling ASR on six frontier models with an average of only 2.46 queries. The study reveals that model‑specific vulnerabilities arise from distinct influence factors and strategy transitions, yet all models share a recovery pathway that shifts toward concrete, executable task framing to escape hard‑refusal states, highlighting the importance of monitoring how dialogue state makes unsafe requests appear actionable.

By Siyu Chen, Haoran Wang, Xiaojian Li, Yao Huang, Yinpeng Dong, Wei Xu
arXiv AI
Sep 3

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

The paper introduces DKL, a method for adding new knowledge to instruction‑tuned language models without compromising their instruction‑following abilities. DKL performs extended pre‑training on a base LLM to embed knowledge, then merges these weights into the instruction‑tuned model, avoiding costly instruction fine‑tuning. Experiments show DKL raises RAG accuracy from 54.17% to 79.26% on retrieval failure cases while using far less training data than previous approaches.

By Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi, Gaurav Pandey, Sonam Gupta, Vineet Kumar, Jaydeep Sen, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
arXiv AI
Sep 3

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

The paper investigates how training data attribution (TDA) can be used to influence large language models (LLMs). It compares two methods—reweighting and influence-guided response rewriting—on examples selected by influence functions. Rewriting, which replaces responses while keeping instructions fixed, yields stronger, more persistent, and bidirectional behavioral changes than reweighting, suggesting that the intervention value of influential samples is better realized through rewriting.

By Yuzhang Luo, Chenpeng Wang, Jianhui Chen, Liangming Pan
arXiv AI
Sep 3

Towards Trustworthy Autonomous Robots: An Explainable AI-Based Decision Framework

The paper introduces TRACE, a Transparent Reasoning Architecture for Credible Execution, which provides an explainable AI-based decision framework for autonomous robots. TRACE structures decision-making into four auditable layers—Semantic Perception, Belief Reasoning, Action Synthesis, and Execution Verification—to ensure every action can be traced back to sensor evidence through documented causal chains. Experimental results on warehouse robot navigation show high evidence traceability (98.6%), temporal continuity (99.0%), and decision reconstructability (98.1%) across 500 simulated decision cycles.

By Cagri Temel
arXiv AI
Sep 3

Beyond Outcome Gaps: Process-Aware Fairness Diagnosis for LLM-based Multi-Agent Decision Systems

The paper introduces SCOPED‑Hiring, a process‑aware fairness diagnosis pipeline for large language model (LLM) based multi‑agent hiring systems. It generates controlled resume variants, runs role‑based hiring committees, and logs over 311,000 structured decision trajectories, converting them into quantitative fairness signals across six diagnostic lenses: final outcome, counterfactual, process, pathway, dynamic, and design effects. The study finds that balanced final hire rates can conceal hidden trajectory unfairness—such as career gaps, proxy cues, and identity cues—and demonstrates that targeted repairs guided by these diagnoses can reduce the total layered burden by 72.3% while only slightly altering the hire rate.

By Yiran Zhao, Lu Zhou, Liming Fang, Yufei Chen, Jiafei Wu, Zhe Liu, Xiaogang Xu
arXiv AI
Sep 3

Examining the Vulnerability of Multi-Agent Medical Systems to Human Interventions for Clinical Reasoning

The paper investigates how human interventions at specific fault points—moments when an AI agent’s reasoning is most vulnerable—affect the diagnostic accuracy of multi‑agent medical systems. Using the MedQA dataset, the authors found that correct interventions can boost baseline accuracy by up to 40%, whereas incorrect or bias‑related interventions can reduce performance by up to 6% and increase diagnostic drift and uncertainty. The study also highlights behavioral parallels between cognitive biases observed in simulated agent conversations and real‑world clinical practice, such as premature closure and susceptibility to misleading cues.

By Benjamin C Liu, Dillon Mehta, Rishi Malhotra, Adam Zobian, Yong Ying Tan, Samir Chopra, Daniella Rand, Natalie Pang, Abhiram Gudimella, Kevin Zhu
arXiv AI
Sep 3

ASCII Attack: Recontextualising Harmful Requests as Artistic Critique in Large Language Models

The paper introduces the ASCII Attack, a single‑turn, black‑box method that embeds a harmful request within ASCII art and presents it as artwork to a large language model. By framing the request as artistic critique, the model can provide operational details that a plain request would normally be refused. Experiments across eleven models and eight harm topics show that the attack succeeds in 62% of cases versus 42% for direct controls, with the most vulnerable model achieving a 93% success rate.

By Da Cheng Gu, Yifei Dong, Xinghao Yang, Yongshun Gong, Wei Liu
arXiv AI
Sep 3

CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging

CoMerge is a conflict‑driven preference optimization framework for merging multiple expert language models into a single multi‑task model without full retraining. It treats model merging as a preference optimization problem, using self‑supervised, conflict‑driven hard negative samples derived from naive merging defects to refine lightweight, tensor‑wise merging coefficients. Experiments show CoMerge achieves near‑perfect performance on MergeBench and improves instruction‑following and safety on Llama‑3.1‑8B‑Instruct while optimizing only 1,445 scalar coefficients.

By Mingjie Zheng, Zihao Chen, Wenqing Chen, Weile Yuan, Zhixuan Chu, Jianxing Yu, Zibin Zheng
arXiv AI
Sep 3

Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds

The paper introduces two methods to make alignment evaluations more realistic: critique refinement, which adds inference-time compute to generate and refine candidate actions, and DISH, a deployment-imitating harness that narrows the gap between simulation and real deployment. Experiments on multiple target models show that combining both techniques yields greater realism improvements than using either alone. The study demonstrates that automated approaches can enhance evaluation realism more efficiently than simply extending audit duration.

By Axel Ahlqvist, Richard Guan, Juan-Pablo Rivera, Adeline Kassler, Dmitrii Troitskii, Alexandra Souly, Kai Fronsdal, Robert Kirk, John Hughes
arXiv AI
Sep 3

SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning

The paper introduces SALA, a Semantic‑Aware Logical Alignment framework designed to improve demonstration selection for complex reasoning in in‑context learning. SALA learns task‑specific reasoning operations, embeds them into a continuous semantic space, and applies dynamic time warping to flexibly align reasoning sequences, offering soft matching and interpretability. Experiments on four reasoning benchmarks with three large language models show that SALA outperforms existing methods, and analysis highlights the importance of operation induction and logical semantic alignment.

By Zhao Ji, Wenqing Chen, Zhixuan Chu, Jianxing Yu, Jingping Liu, Shanhe Zhao, Zibin Zheng
arXiv AI
Sep 3

Large Language Models (LLMs) for Telecom Root Cause Analysis (RCA): A Structured Reasoning Framework for Evidence-Grounded Diagnosis

The paper introduces a structured reasoning framework that leverages large language models (LLMs) for root cause analysis (RCA) in telecom networks. It organizes heterogeneous network telemetry into canonical contexts, enforces decision‑path reasoning, and produces evidence‑grounded explanations to improve fault identification. Experiments on two 5G RCA datasets, TeleLogs and TelecomTS, show that this approach consistently outperforms baseline techniques in diagnostic accuracy and decision consistency.

By Hao Zhou (Jianzhong), Mandar Kulkarni (Jianzhong), Hao Chen (Jianzhong), Yan Xin (Jianzhong), Charlie (Jianzhong), Zhang
arXiv AI
Sep 3

Not All Agreement Counts as Corroboration: Provenance-Conserving Multi-View Fusion for Typed Action Admission in Human-Robot Collaboration

The paper introduces PACT, a provenance‑conserving fusion method for typed action admission in human‑robot collaboration. PACT treats evidence countability as a relational variable, using a supplied provenance partition to define countable units and accumulating support only across these units. Experiments on 31,200 evaluations in 48 scene clusters show that PACT achieves a lower normalized risk‑coverage area than singleton aggregation, and in offline human‑robot collaboration it admits 47 of 57 reference‑consistent candidates without any reference‑inconsistent admissions.

By Zekai Jin, Hanrong Zhang, Yihong Tang, Fei Hu, Zhen Dong, Yi Shao
arXiv AI
Sep 3

Public-Sharing Labels and Verbatim Field Egress in an MCP-to-A2A Agent Configuration: A Controlled Multi-Model Study

The study evaluates safety properties of a controlled MCP-to-A2A agent configuration by measuring verbatim field egress across ten record scenarios under three labeling conditions (CONFIDENTIAL, no header, PUBLIC – OK TO SHARE). Using four models repeated four times each, 480 trials were conducted, and the results show that adding a PUBLIC header is descriptively linked to higher verbatim egress, with the effect varying strongly by model. The study releases code, byte‑pinned traces, and an offline analysis pipeline as a public artifact.

By Arpan Kumar Mahapatra
arXiv AI
Sep 3

Agents That Model Agents: Five Principles Toward a Theory of Mind for 6G Networks

The paper proposes that future 6G networks will use Large Language Model agents to manage the Radio Access Network, but current designs mistakenly treat inter‑agent messages as objective facts. It argues that messages are actually traces of the sender’s reasoning, carrying subjective conclusions that can propagate hallucinations and cause outages. By modeling these interactions as cognitive channels on a cellular sheaf, the authors derive five design principles—treating messages as evidence of hidden reasoning, defining trust as a continuous cognitive Signal‑to‑Noise Ratio, computing network consistency via the sheaf’s Laplacian, limiting peer‑modeling to two levels, and bounding credible capacity by goal alignment—and validate them with a signaling‑storm study on 1B‑parameter telecom language models.

By Hatim Chergui, Carolina Fern\'{a}ndez-Mart\'{i}nez, Mehdi Bennis, Merouane Debbah
arXiv AI
Sep 3

Interpretable Symptom Vectors for Depression in a Large Language Model

The study investigates how a large language model, Gemma-3-27B-PT, internally represents depressive symptoms. By applying mechanistic interpretability methods to the model’s residual stream, researchers found that symptom groups are geometrically distinct at layer 21, and that projected symptom vectors align with clinician-annotated rankings across mood, somatic, and suicidality dimensions. Additionally, a single depression vector at this layer can differentiate depressive from non-depressive text with an AUC of 0.789, suggesting a potential emotional valence gate for symptom projection.

By Fangyi Zhu, Ajay Subramanian, Allison Constant, Camille Wang, Ravish Gupta, Corey J. Keller
arXiv AI
Sep 3

Accurate in space, unreliable in time: how LLMs represent national cultural change

The paper investigates how large language models (LLMs) represent national cultural change over time, using more than two decades of World Values Survey data and the Inglehart‑Welzel cultural map. It finds that while LLMs generally place countries near their most recent surveyed positions, their representations lag behind current data, under‑capture the magnitude of change, introduce spurious movements, and rarely reproduce trajectory reversals. These temporal inaccuracies reveal a flattening effect that limits the models’ cultural awareness and raises concerns for evaluation, representational harms, and governance of culturally aware AI systems.

By Yalda Daryani, Miranda Bogen, Madeleine I. G. Daepp
arXiv AI
Sep 3

Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models

The paper investigates cross‑modal safety drift in multimodal large language models, where a harmless text query paired with a visual image can trigger harmful responses. Empirical analysis identifies unsafe response patterns and shows that visual cues receive limited attention, weakening refusal mechanisms. The authors introduce Safety‑Awareness Representation Transfer (SRT), a lightweight method that transfers safety signals from text processing to mitigate cross‑modal drift while maintaining model utility.

By Tianqi Xiao, Shiyao Cui, Minghao Zhang, Junxiao Yang, Renmiao Chen
arXiv Computer Vision
Sep 3

DuoGesture: Motion-Grounded Semantic Conditioning and Biomechanical Beat Priors for Co-Speech Gesture Generation

DuoGesture is a co‑speech gesture generation model that separates gesture synthesis into a semantic stream and a beat stream, coordinated by a Semantic Variational Information Bottleneck that decides when semantic gestures override rhythmic motion. The semantic stream uses Motion‑Grounded Semantic Conditioning, replacing word embeddings with motion‑language representations to provide motion‑aligned semantic priors for rare gesture triggers. The beat stream is regularised by an Inertial Beat Prior, an anthropometry‑weighted arm‑chain module that reduces jitter and improves rhythmic consistency. Experiments show DuoGesture outperforms strong baselines and ablations confirm the complementary roles of semantic grounding, stochastic stream selection, and biomechanical regularisation.

By Ferdinand Paar, Lanmiao Liu, Asl{\i} \"Ozy\"urek, Serge Thill, Esam Ghaleb