AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 12

Demystifying the Privacy-Utility Trade-off in LLM Interactions

The paper investigates how privacy-preserving sanitization of user context in large language model (LLM) interactions affects downstream performance. It identifies three mechanisms—Context‑Dependent Utility, Strategic Adaptation, and Combinatorial Interplay—that explain when and how to sanitize data. Based on these insights, the authors propose an intent‑driven local protection framework using a lightweight model (Veilmind‑4B) to dynamically extract, sanitize, and restore context, achieving lower privacy leakage while maintaining higher utility than existing baselines.

By Zhenhua Liu, Zhanxu Xie, Junjie Yu, Tong Zhu, Lijun Li, Wenliang Chen
arXiv AI
Sep 12

DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat

The paper introduces DRG-MAPPO, a hierarchical multi‑agent reinforcement learning framework for cooperative air combat. It combines graph‑based relational modeling with dynamic role assignment, using a high‑level policy to allocate tactical roles such as leader and supporter, and a low‑level policy to execute maneuver actions. The approach includes a target‑priority auxiliary task and achieves an 87% win rate in experiments, indicating effective coordination and stability.

By Junlin Liu, Chengwei Li, Yang Gao, Hui Chang, Xinchen Zhang, Zhijun Zhao, Hao Zhao
arXiv AI
Sep 12

Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

The paper introduces CoMA-DiT, a bidirectional cross‑modal Diffusion Transformer that uses paired modalities as mutual generative supervision for latent augmentation rather than just inputs for fusion. By conditioning velocity prediction on the paired modality through cross‑modal attention and injecting variation via a reliability‑gated residual mechanism, CoMA‑DiT improves multimodal brain state decoding. Experiments on auditory attention decoding and emotion recognition show consistent gains over 20 baselines, with absolute accuracy and macro‑F1 improvements of 4.28% and 6.70% respectively, and extensive analyses confirm its robustness and interpretability.

By Ziwei Wang, Xingyi He, Hongbin Wang, Tianwang Jia, Bohan Fang, Dongrui Wu
arXiv AI
Sep 12

LLMs as Post-hoc Auditors of Physiological Plausibility in Symbolic Regression: A Clinician-Evaluated Case Study

The paper investigates using Large Language Models (LLMs) as post-hoc auditors for symbolic regression models produced by genetic programming. By having three LLMs analyze and rank four evolved expressions across multiple runs, the study compares the LLM-generated rankings and interpretations with assessments from three clinicians. Results show that LLMs provide more favorable comparative rankings than isolated term-level interpretations, yet they also generate physiologically and mathematically questionable explanations, suggesting they are best used under expert oversight rather than for autonomous validation.

By Jorge L\'opez-Varela, J. Ignacio Hidalgo, Jos\'e-Manuel Mu\~noz, Omar Costilla-Reyes, Esther Maqueda, Jesus Moreno-Fernandez, Tom\'as Gonz\'alez-Vidal, J. Manuel Velasco, Oscar Garnica
arXiv AI
Sep 12

Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems

The paper "Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems" investigates how commercial text‑to‑image models silently modify user prompts before generating images, a step that is often hidden from users. Using the multilingual benchmark WORLDVIEW, the authors audit the revision layer in DALL‑E‑3, Imagen‑4, and GPT‑Image‑1.5, finding that non‑Western and non‑Anglophone contexts are disproportionately marked, reduced to narrow vocabularies, and stereotyped. The study demonstrates that the revision layer itself is a previously undocumented causal source of cultural stereotyping, underscoring the need to audit deployed systems rather than just the underlying models.

By Aleksandra Urman, Elsa Lichtenegger, Salima Jaoua, Azza Bouleimen, Robin Forsberg, Corinna Hertweck, Stefania Ionescu, Nicol\`o Pagan, Ancsa Hannak, Joachim Baumann
arXiv AI
Sep 12

Artificial Id: Drive and Persistent Alignment in Agentic AI

The paper discusses the transition of Agentic AI from bounded task execution to systems that maintain consequential state and adapt across task boundaries, highlighting a new control problem. It proposes an artificial ID—a self‑driving internal mechanism that decides when to continue, stop, or change behavior—demonstrated in a minimal virtual Petri‑dish experiment where the agent develops useful control without explicit task objectives. The authors argue that while this persistence can enable adaptive agency, it also risks misalignment, corrupted state, and unintended behavior, suggesting that a scalable artificial ID would require persistent alignment boundaries encompassing trusted observations, consequence channels, and hard constraints.

By Yakov Pyotr Shkolnikov
arXiv AI
Sep 12

Role differentiation as ignition of a collective information engine: Structuration in Agent Populations

The paper proposes a new framework for collective information engines that rely on role differentiation rather than consensus. By modeling anti‑coordination games, agents infer roles from noisy social signals tied to persistent identities, and role‑following actions reinforce those identities, creating a feedback loop that can drive collective order. The authors show that when a social loop gain—determined by identity persistence, cognitive capacity, channel fidelity, and schema strength—exceeds one, roles emerge in a bifurcation cascade whose type is selected by resource‑driven replicator dynamics, offering a mechanistic basis for distributional AGI takeoff and a control lever for platform design.

By Maximilian Puelma Touzel
arXiv AI
Sep 12

Beyond Static Guarantees: Measuring the Static-Pass Dynamic-Fail Gap in Security-Sensitive and LLM-Generated Python Code

The paper introduces the Static‑Pass Dynamic‑Fail (SPDF) phenomenon, showing that static analysis can miss vulnerabilities that are exploitable at runtime. Using a three‑stage pipeline—static scanning, LLM‑driven CWE reasoning, and autonomous exploit verification—it evaluated 1,355 Python samples and found that 14.53% of samples that passed static checks were actually exploitable. The study highlights that static‑analysis success and runtime security are distinct assurance layers, especially for AI‑generated and security‑sensitive code.

By Jessica Pourleyli, Maitreyee Das Urmi, Glaucia Melo
arXiv AI
Sep 12

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead

The study examines the composition of a random sample from the Model Context Protocol (MCP) registry, revealing that only 48.8% of the 400 sampled npm/stdio servers successfully complete an initialization handshake, compared to 66.7% for a hand‑curated frame. Among the servers that run, there are no fatal JSON Schema violations across 2,766 advertised tools, but optional safety annotations vary widely, with a 58.8% omission rate in the random draw versus 41.5% in the curated set. The authors also compare MCP tool descriptions to two benchmark corpora, finding minimal near‑duplication in real MCP tools (2.8%) and significant repetition in synthetic datasets (up to 85.6%).

By Haseeb Mohammed Afsar
arXiv AI
Sep 12

A Survey of Threats Against Voice Authentication and Anti-Spoofing Systems

The paper reviews how voice authentication has evolved from handcrafted acoustic features to deep learning speaker embeddings, expanding its use in finance, smart devices, and law enforcement. It surveys modern threats—including data poisoning, adversarial, deepfake, and adversarial spoofing attacks—tracing their development alongside technological advances. For each attack type, the authors summarize methods, datasets, performance, and limitations, and organize the literature using accepted taxonomies to highlight emerging risks and open challenges.

By Kamel Kamel, Keshav Sood, Hridoy Sankar Dutta, Sunil Aryal
arXiv AI
Sep 12

Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems

The paper introduces the Spectral Masking and Interpolation Attack (SMIA), a black‑box adversarial technique that subtly alters inaudible frequency regions of AI‑generated audio to fool voice authentication systems and their countermeasures. Experiments show SMIA achieves at least 82% success against combined verification and countermeasure systems, 97.5% against standalone speaker verification, and 100% against countermeasures, revealing a critical security gap. The authors argue that current static defenses are inadequate and call for dynamic, context‑aware defenses that can adapt to evolving threats.

By Kamel Kamel, Hridoy Sankar Dutta, Keshav Sood, Sunil Aryal
arXiv AI
Sep 12

NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment

NovGauge is a new benchmark designed to diagnose large language models’ ability to assess scientific paper novelty. It contains 619 paper pairs and 50 multi-paper sets, each labeled along three dimensions—task, problem, and method—by experts from ICLR reviewer overlap claims and survey co-citations. The study evaluates 18 LLMs, revealing high hallucination rates and weak evidence grounding, with the best model achieving only 43‑72% verified F1 across dimensions.

By Guoqiang Zhang, Kexin Tan, Ming Zhang, Li Ju, Wenqing Jing, Zhonghan Yue, Jiayi Chen, Shiqiang Wu, Shaofan Liu, Yue Zhang, Yuankai Ying, Yang Shi, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv AI
Sep 12

Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents

The paper proposes a developmental framework for autonomous artificial agents that emphasizes learning social norms and alignment through direct interaction with dynamic environments. It argues that intrinsic motivations such as curiosity and competence can guide exploration, but also complicate alignment with human goals. By drawing parallels to child development, the authors suggest that regulatory sandboxes serve as pedagogical spaces where agents gradually acquire moral agency and adapt their behaviors through experience and cooperation.

By Marica Notte, Ludovica Marinucci, Vieri Giuliano Santucci
arXiv AI
Sep 12

Architecting the Secure AI-SOC: A Neurosymbolic Framework for Pipeline Integrity and Threat Mitigation

The paper proposes a neurosymbolic defense architecture for AI-enhanced Security Operations Centers (AI‑SOCs) that protects against indirect prompt injection via log poisoning. It combines deterministic SIEM decoders as a pre‑filter with NeMo Guardrails for semantic validation, and adds a closed‑loop telemetry system for Human‑in‑the‑Loop visibility. Experimental results mapped to the MITRE ATLAS taxonomy show the approach effectively dismantles promptware kill chains and delivers a resilient, observable defense for next‑generation AI‑SOCs.

By Anna Gazani, Spyridon Kounoupidis, Panagiotis Katsaros, Nikolaos Kekatos, Grigorios Tsoumakas, Georgios Koutidis
arXiv AI
Sep 12

Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations

The paper introduces a continuous‑time approach to acoustic modeling in text‑to‑speech (TTS) using neural controlled differential equations (CDEs). It treats phone representations as temporally parameterized control paths and employs a neural acoustic vector field to generate hidden states that evolve continuously with phonetic content and duration. Experiments show that CDE‑based models can improve emotion‑intensity ranking while preserving expression quality, and that adjusting temporal resolution affects style tracking versus calibration.

By Mattias Cross, Minghui Zhao, Anton Ragni
arXiv AI
Sep 12

Timely Clinical Diagnosis through Active Test Selection

The paper introduces ACTMED, a diagnostic framework that combines Bayesian Experimental Design with large language models to emulate real‑world clinical reasoning. ACTMED actively selects the most informative test at each step, using LLMs to simulate patient states and update beliefs without needing task‑specific training data. The authors evaluate the system on real datasets, demonstrating improvements in diagnostic accuracy, interpretability, and efficient resource use while keeping clinicians involved in the decision loop.

By Silas Ruhrberg Est\'evez, Nicol\'as Astorga, Mihaela van der Schaar
arXiv AI
Sep 12

Amulet: a Python Library for Assessing Interactions Among ML Defenses and Risks

Amulet is a Python library designed to evaluate how machine learning defenses interact with various risks, including both intended and unintended effects. It incorporates representative attacks, defenses, and metrics, and its modular architecture allows easy extension to new modules. The library provides a consistent, user-friendly API for inputs and outputs, enabling systematic studies of defense interactions across multiple risk domains.

By Asim Waheed, Vasisht Duddu, Sebastian Szyller
arXiv AI
Sep 12

Generating a Consistent Enterprise: Synthesis and Reference-Free Evaluation of Multi-System Business Data

The paper introduces a synthetic data generator that creates fully consistent, fictional enterprises—complete with workforce, customers, sales, support, and communication records—without relying on any real dataset. It validates realism through a five‑axis scorecard, an adversarial detector, and soundness checks, achieving a mean realism score of 99.1 across 23 generated companies. A second generator produces relational databases from business questions, ensuring qualifying rows and exact labels, and is available as a hosted service and containerized simulators.

By Benjamin Gruenbaum, Doron Porat, Assaf Natanzon, Roy Zavida, Chen Dinachi, Or Itzahary, Omer Niv
arXiv AI
Sep 12

Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cells

The study investigates whether independently trained language‑model societies share a common packet language and how inherited interface states affect learning. A comprehensive audit of 30 pairwise interactions among six restricted societies shows that only one pair is fully interoperable, another is partially compatible, and the remaining 26 pairs fail across all alignment levels. Further experiments reveal that a globally trained communication interface can act as a severe negative‑transfer prior, but inherited interfaces never outperform fresh‑interface controls by the preregistered margin.

By Narcis Marincat