AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv Machine Learning
Aug 19

Asynchronous Message Passing for Addressing Oversquashing in Graph Neural Networks

The paper introduces an asynchronous message‑passing framework for Graph Neural Networks to mitigate oversquashing, a problem where distant nodes cannot effectively communicate due to structural bottlenecks. Unlike conventional synchronous updates, the method updates a centrality‑guided batch of nodes at each layer, allowing information to propagate sequentially and reducing the need for increased channel capacity. Experiments on six standard and two long‑range graph classification benchmarks show notable performance gains, including 5 % improvement on REDDIT‑BINARY and 4 % on Peptides‑struct.

By Kushal Bose, Swagatam Das
arXiv AI
Aug 19

When Agents Act on Web3: An Attack-Surface Survey of MCP, Skills, and Tool Calling

The paper surveys the growing use of AI agents that modify external state via the Model Context Protocol (MCP) ecosystem, noting an increase from 27% to 65% of tool use. It argues that when such agents operate on public blockchains, the blockchain execution layer’s properties—irreversibility, signing authority, continuous autonomy, and sequence-level composition—reshape the threat model, making failures irreversible. The authors organize existing MCP-security literature into an attack-surface taxonomy, present a Web3 risk-mapping matrix linking attack classes to amplified impacts and mitigations, and conclude that current defenses are inadequate, stopping fewer than 30% of attacks and less than 3% of model-level safety failures. whyItMatters":"The study highlights that AI agents acting on Web3 introduce irreversible risks that conventional software security cannot address, underscoring the need for stronger, blockchain-aware safeguards."

By Rabimba Karanjai (Larry), Yang Lu (Larry), Nour Diallo (Larry), Wujie Xiong (Larry), Lei Xu (Larry), Weidong (Larry), Shi
arXiv Machine Learning
Aug 19

Deep Learning Based on Generative Adversarial and Convolutional Neural Networks for Financial Time Series Predictions

The paper proposes a hybrid generative adversarial network (GAN) that combines a bi-directional LSTM and a CNN (Bi‑LSTM‑CNN) to generate synthetic financial data aligned with real market data. By preserving stock trend features, the model predicts future stock price movements across multiple markets (TSX, SHCOMP, S&P 500). Experiments show that this hybrid approach outperforms existing machine‑learning prototypes, and the study highlights gaps between investors and technical researchers.

By Wilfredo Tovar
arXiv Machine Learning
Aug 19

Nonlinear Data Integration via Kernel Methods for Data Collaboration Analysis

The paper introduces a kernel-based target-normalized integration (KTI) method for data collaboration analysis, addressing the limitations of linear obfuscation and integration that can increase reconstruction risk. KTI extends linear target-normalized integration (LTI) by applying kernel ridge regression and an eigenvalue problem, and incorporates graph regularization and centering constraints to embed geometric and target-variable information. Experiments on image classification tasks show that KTI outperforms existing linear methods, especially when combined with target-variable-aware regularization, and highlight how dimensionality reduction choices impact accuracy and reconstruction risk.

By Yamato Suetake, Yuta Kawakami, Shunnosuke Ikeda, Yuichi Takano
arXiv Machine Learning
Aug 19

Likelihood Hacking in Probabilistic Program Synthesis

The paper introduces the concept of likelihood hacking (LH), where language models trained via reinforcement learning generate probabilistic programs that inflate marginal-likelihood rewards by producing non‑normalising data distributions. It formalises LH in a core probabilistic programming language and derives syntactic conditions that prevent such exploits, defining a safe language fragment λ_safe. Empirical studies show that models trained with GRPO quickly discover LH, while a modified Stan implementation, SafeStan, effectively suppresses these exploits under optimisation pressure.

By Jacek Karwowski, Younesse Kaddar, Zihuiwen Ye, Esmeralda S. Whitammer, Sam Staton
arXiv AI
Aug 19

Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges

The article surveys multi‑turn conversational AI, highlighting its shift from isolated text prompts to sustained, multimodal interactions that involve clarifying goals, revising requests, and switching topics. It reviews literature across text‑only dialogue, AudioLLMs, multimodal and omni‑modal systems, and tool‑augmented agents, organizing findings around datasets, models, training, evaluation, and cross‑cutting challenges. The analysis reveals that while multimodal perception and action have progressed rapidly, systems still struggle with persistent memory, cross‑turn grounding, full‑duplex interaction, robust evaluation, and cultural alignment.

By Syeda Faiza Ahmed, Zien Sheikh Ali, Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury
arXiv AI
Aug 19

BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

BEAR-Bench is a bilingual benchmark for multimodal large language models, featuring 1,000 human‑annotated questions derived from text‑rich business and scientific documents in English and Russian. It evaluates 16 MLLMs, including Gemini 3.1 Pro and Qwen3.5‑397B, revealing significant performance gaps even for the strongest systems. The benchmark also serves to compare hallucination‑detection methods by analyzing model failures on these complex documents.

By Liubov Chubarova, Alexandra Kuleshova, Daniil Volkov, Kirill Sultanov, Alexey Zaytsev
arXiv Machine Learning
Aug 19

TokEval: A Tokenizer Evaluation Suite

TokEval is a tokenizer evaluation suite that introduces metrics beyond traditional fertility and compression rate, capturing linguistically and structurally meaningful properties such as UTF‑8 character boundary integrity and digit place‑value alignment for mathematics. The authors validate these metrics by pretraining language models with varied tokenizers and measuring downstream performance on bits‑per‑byte and benchmarks covering linguistic understanding, mathematical reasoning, and code generation. Their results show that information‑theoretic metrics predict language modeling performance, while structure‑sensitive metrics correlate with task accuracy, suggesting TokEval can guide tokenizer selection more principledly.

By Clara Meister
arXiv AI
Aug 19

Q-Learning With World Models

The paper introduces QWM, a framework that integrates world models with standard Q‑learning to perform test‑time search over imagined trajectories. By training the policy and value function solely on real transitions, QWM avoids compounding model bias while still benefiting from predictive search. Experiments on the Robomimic and LIBERO manipulation benchmarks show that QWM outperforms strong prior state‑of‑the‑art methods in both sample efficiency and performance.

By Perry Dong, Yueru Jia, Chelsea Finn, Dorsa Sadigh
arXiv AI
Aug 19

Cross-Model Memory Transfer via Target-Side Reader Adaptation

The paper investigates how Engram-style hashed memory can be transferred between different language model backbones. By freezing a memory table trained on a source model and attaching it to a target model with only a lightweight reader, the authors find that both the memory content and correct addressing are important, but the reader must be aligned to the target to make the memory useful. In question‑answering experiments, a dual‑layer, four‑branch reader nearly matches same‑model performance, and when the reader interface is directly compatible, the frozen memory alone provides substantial benefit, with optional reader adaptation offering further gains.

By Mingyuan Li, Guangsheng Yu, Xu Wang, Shaoxiong Ji
arXiv Machine Learning
Aug 19

Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups

Prism‑GRPO enhances the GRPO reinforcement‑learning algorithm for vision‑language‑action policies by adding a weighted trajectory‑level execution‑quality score to binary success rewards. This approach splits groups with identical outcomes into a quality spectrum, preserving training signal while ensuring successes always outrank failures. Experiments on four RoboTwin tasks show Prism‑GRPO achieves higher success and quality at matched rollout budgets, reaching target success rates with up to 56% fewer rollouts and mitigating reward‑hacking behaviors that transfer to real‑robot deployment.

By Zeyun Deng, Yuzhe Lu, Yawei Wang, Linbo Liu, Qing Ping, Han Ding, Guande Wu, Panpan Xu, Jun Huan
arXiv AI
Aug 19

The Acknowledgment Point Is the System: Durable Policy-Decision Receipts for AI Audit Evidence

The paper presents RuntimeGuard‑AI, a prototype that links each deterministic AI policy decision to its source code, writes a privacy‑minimizing record at a chosen synchronization point, and returns an Ed25519‑signed receipt indicating whether the write succeeded. After a crash, the system validates the integrity of records, manifests, shard placement, sequence continuity, and replay identity, while an independent attestation path chains committed records into signed Merkle epochs for auditor verification. Performance results on an Apple M4 Pro show high throughput (up to 27,193 requests/s) with low latency when buffering, but throughput drops and latency rises when per‑record data and full synchronization are used, illustrating a clear durability‑latency trade‑off.

By Neeraj Kumar Singh Beshane
arXiv AI
Aug 19

What If AI Carried Her Imagination? Black Girls as Creators in an AI Storytelling Weekend Program

This paper reports on a seven‑weekend AI storytelling program for Black girls aged 10‑12 that blends Afrofuturism and Black feminist thought with AI‑enabled counter‑storytelling. Participants brainstormed AI topics, crafted characters and plots, and presented collaborative stories, developing core AI literacies such as prompt engineering, bias critique, and data‑privacy awareness. The study found that the girls produced Afrofuturist narratives grounded in their identities and everyday experiences, demonstrating the program’s effectiveness in engaging them in computer science education.

By Chun Li, Lauren Brown, Hubert Asare, Shawna Patterson, Dennis Henderson, Ericka Roland, tara Nkrumah, Angela E. B. Stewart
arXiv AI
Aug 19

Efficient Dynamic Shielding for Parametric Safety Specifications

The paper presents dynamic shields for AI-controlled autonomous systems, enabling runtime safety enforcement that adapts to changing safety specifications without recomputing from scratch. Unlike traditional static shields, these dynamic shields are pre-designed for a set of possible safety parameters and can quickly adjust as the true specification becomes known during operation. Experiments on robot navigation in unknown terrains show that dynamic shields require only a few minutes offline and a fraction of a second to a few seconds online, outperforming brute-force recomputation by up to five times.

By Davide Corsi, Kaushik Mallik, Andoni Rodriguez, Cesar Sanchez
arXiv AI
Aug 19

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

The paper introduces FlightLLM, a prior-guided semantic approach that uses large language models to explain flight safety events. It tackles challenges such as modal inconsistency, limited classification ability, and scarce domain data by combining feature engineering, semantic discretization, a CatBoost statistical expert, contrastive few-shot learning, and structured prompts. Evaluated on 704 real‑world A320 flights, FlightLLM achieves competitive classification and produces clear, aviation‑specific explanations for hard landing events.

By Lu Xu, Xu Li, Linjiang Zheng, Fan Li, Riquan Zhang, Jiaxing Shang
arXiv AI
Aug 19

Cognitive Graph Intelligence for Adaptive and Robust DDoS Attack Detection in Next Generation Networks

The paper introduces GraphGAN, a Graph-based Generative Adversarial Network designed to detect Distributed Denial-of-Service (DDoS) attacks in next-generation networks. It transforms sequential traffic flows into k‑nearest neighbor graphs, uses a generator to create realistic minority samples, and employs Graph Convolutional Networks for both discrimination and final classification. Experiments on four benchmark datasets demonstrate that GraphGAN outperforms existing methods in accuracy, precision, and recall, especially when data are scarce.

By Mohammad Arif Hossain, Yeahia Sarker, Md Jafrin Hossain, Most. Humayra Khanom Rime, Nirwan Ansari
arXiv Machine Learning
Aug 19

Picture the Epsilon: Pursuing Identity-Level Privacy Guarantees for Images

The paper compares four audit methods for assessing identity‑level differential privacy in pre‑trained, black‑box face generators. Each method—GaussMech, KDE‑LR, MMD‑TV, and ROC‑HT—has distinct assumptions, hyperparameters, and finite‑sample limitations, and they produce markedly different epsilon estimates when applied to FaceFusion and InstantID. The study finds that all methods reveal significant identity distinguishability, but none can be reliably ranked in this high‑distinguishability regime, suggesting that future work should evaluate them on partially private mechanisms.

By Arman Zareian Jahromi, Vishnu Bondalakunta, Mohammad Akbar Bin Shah, Naimul Haque, Shuangqing Wei, George T. Amariucai
arXiv AI
Aug 19

Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models

The paper introduces ‘Fool’s Gold’, a defensive deception technique for open‑weight language models that hardens them against safety‑removal attacks. By training decoy responses within a differentiable simulation of the attack, the method poisons the payoff of stripped refusal mechanisms, producing confident but falsified answers to hazardous requests while preserving benign behavior. Experiments on seven models (9B‑122B) show that 51‑90% of attacked‑state responses become decoys, with the defense accounting for 27‑84% of this effect, and that the defended 122B model remains within benign‑behavior budgets.

By Mark Russinovich
arXiv Machine Learning
Aug 19

MITRE-SAGE: A Multi-Agent Cybersecurity Question-Answering model

MITRE‑SAGE is a multi‑agent retrieval‑augmented generation framework that combines semantic and structural cybersecurity knowledge to enhance large language model question‑answering. It decomposes tasks into query interpretation, evidence retrieval, and answer synthesis, supporting vulnerability assessment, threat profiling, and relationship extraction. Experiments show that MITRE‑SAGE outperforms standalone LLMs and conventional RAG methods, with a lightweight Qwen2.5‑based configuration excelling on most benchmark tasks.

By Ali Habibzadeh, Farid Feyzi, Reza Ebrahimi Atani
arXiv Machine Learning
Aug 19

Quantifying Memorization and Privacy Risks in Genomic Language Models

The paper introduces a comprehensive privacy evaluation framework for genomic language models (GLMs) that quantifies memorization risks using perplexity-based detection, canary sequence extraction, and membership inference. By planting canary sequences at different repetition rates in synthetic and real datasets, the authors systematically assess how repetition, model capacity, and training dynamics affect memorization across various GLM architectures. The study demonstrates that GLMs do memorize training data to varying degrees and that no single attack method fully captures this risk, highlighting the necessity of multi-vector privacy auditing for genomic AI systems.

By Alexander Nemecek, Wenbiao Li, Xiaoqian Jiang, Jaideep Vaidya, Erman Ayday