AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Oct 2

Gradient-Aligned Pair Selection for Personalized Preference Optimization

The paper introduces GAP-DPO, a method for personalizing large language models by selecting preference pairs based on gradient alignment with user utility. It formalizes personalized preference learning as a geometry‑aligned optimization problem, showing that off‑policy sampling can shift DPO updates from error correction to reinforcement when preference margins align with utility gradients. Experiments demonstrate that GAP‑DPO improves stylistic fidelity, preference alignment, and overall generation quality over standard DPO variants.

By Ruoming Jin, Xinyu Li, Hao Zhou, Jianfeng Zhu, Ruixin Guo, Feodor Dragan, Lei Xu, Haixun Wang, Yang Zhou
arXiv AI
Oct 2

Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts

The study evaluates how scientific explanations influence experimental forecasts by comparing paired predictions that share the same intervention, forecaster, and outcome but differ in description, matched explanation, and donor context. Across 336 prospective states, 12 Tox21 endpoints, and 24 OpenML tasks, the predictive credit of explanations was largely inconclusive, with key metrics such as ROC AUC interval-score and point-equivalence rules unmet. While certain model variants (e.g., DeepSeek V4 Pro) reduced drift and improved some error metrics, overall matched point-accuracy gains and natural-explanation credit remained unconfirmed at the tested donor resolutions.

By Jingjie Ning, Xueqi Li, Yibo Kong, Dongting Li
arXiv AI
Oct 2

An ontology for cross-sectoral crisis management: core and public health modules

The paper introduces the European Crisis Management Ontology (ECMO), an OWL-based modular ontology designed to serve as a cross‑sectoral reference for disaster risk reduction and response. ECMO-CORE contains core crisis management concepts such as hazard, event, exposure, impact, and response measure, employing ontology design patterns and OWL2 punning to clarify ambiguities. Domain‑specific modules, including a public health module aligned with SNOMED CT and ICD‑11, extend the ontology; a demonstration shows ECMO can transform unstructured epidemiological news into a compliant knowledge graph, illustrating its potential for consistent knowledge integration.

By Aldo Gangemi, Rita T. Sousa, Luigi Asprino, Giorgia Lodi, Andrea G. Nuzzolese, Valentina Presutti, Johannes Gysen, Diana F. Sousa, Luigi Spagnolo
arXiv AI
Oct 2

vFedProtoQNAS: Prototype-Guided Personalized Quantum Neural Architecture Search for Virtual Federated Learning

The paper introduces vFedProtoQNAS, a prototype‑guided personalized quantum neural architecture search method for virtual federated learning. It allows each client to independently design and train a device‑specific quantum neural network while avoiding parameter aggregation across structurally different models. Instead, clients share class‑wise latent prototypes, which are refined using global prototypes from the server to serve as federated semantic anchors, leading to a 3.70% accuracy improvement over FedAvg.

By Seok Bin Son, Samuel Yen-Chi Chen, Soohyun Park, Joongheon Kim
arXiv AI
Oct 2

Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints

The paper introduces the Homomorphic Advantage Operator (HAO), a stabilization framework for reinforcement learning under fully homomorphic encryption (FHE). HAO applies a zero‑mean centering projection to temporal‑difference targets, eliminating the uniform state‑value baseline that causes the Bellman drift and keeping network pre‑activations within safe polynomial approximation bounds. Experiments on tabular MDPs, an encrypted CartPole environment, and a logistics routing benchmark show that HAO prevents boundary breaches entirely, improves optimal policy accuracy by 18 percentage points, and remains stable even with added Gaussian noise.

By Abid Mohamed Nadhir, Ahmad Al Hanbali, Beggas Mounir
arXiv AI
Oct 2

Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights

The paper evaluates legal text classification models for Korean sexual offense cases, comparing traditional machine learning, large language models, and fine‑tuned domain models. Fine‑tuned KLUE‑BERT achieved the highest accuracy of 99.3%, outperforming GPT‑3.5, GPT‑4.0, and other traditional approaches. Explainable AI techniques were used to analyze predictions, revealing linguistic features that influence decisions and highlighting limitations in capturing subtle textual cues, especially in real‑world KICS data.

By Jeongmin Lee
arXiv AI
Oct 2

Tokenized Key-Gated Adapter Routing: A Secure Access Control Mechanism Against Private Data Leakage in LLMs

Tokenized Key-Gated Adapter Routing (Locket) is a framework that embeds fine‑grained, policy‑driven access control into large language models by training lightweight LoRA adapters for different privacy policies. A gating module associates a learned keyed entry token with a specific adapter, allowing authorized tokens to unlock private knowledge while invalid or missing tokens trigger privacy‑preserving adapters that redact or sanitize sensitive content. Experiments on datasets such as Enron, ECHR, and Yelp with models like Qwen3, Llama‑3.2, and Gemma‑2‑2B show that Locket maintains perplexity comparable to fine‑tuning when the correct token is provided, and significantly reduces PII leakage when the token is absent or invalid, without sacrificing utility.

By Mohamed Shaaban, Mohamed Elmahallawy
arXiv AI
Oct 2

DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents

DuplexSpeechBench-Document Grounding (DSB‑DG) is a benchmark that evaluates how well voice agents can ground their responses in external documents across five professional domains. It focuses on three failure modes—Context Saturation, Grounding Decay, and Proactive Grounding—and includes 1,636 adversarially verified QA pairs from 50 documents. The benchmark measures grounding accuracy, hallucination, and response latency, revealing that cascaded pipelines achieve the highest accuracy while open‑weight systems suffer from abrupt capacity collapse and multi‑turn decay.

By Puneet Mathur, Nedim Lipka, Zeyu Jin, Dinesh Manocha
arXiv AI
Oct 2

When the Judge Acts: Auditing VLM-Guided Image Selection on Culturally Situated Prompts

Vision‑language models (VLMs) are increasingly used as judges to select the best image from a set of generated options, yet their decisions are usually validated only by score agreement with human ratings rather than by the images they return. In this study, the authors audited VLM judges on 300 culturally situated prompts, comparing the returned image with unseen human ratings and with random choice, and repeated each decision with reordered candidates. They found that a 4‑B parameter judge barely outperforms random selection, shows significant position bias, and is highly sensitive to candidate ordering, while an 8‑B judge reduces position bias and outperforms a CLIP similarity baseline but still discards many good decisions when filtered.

By Huichan Seo
arXiv AI
Oct 2

Multi-Party Backchannel Prediction: a Diagnosis, a Benchmark, and a Ceiling

The paper introduces a multi‑party backchannel prediction benchmark built from the AMI meeting corpus, featuring 682 masked‑listener views, 190 speakers, and 18,697 backchannel events. A state‑of‑the‑art dyadic model performs at chance when applied zero‑shot to meetings, but its frozen acoustic features are still informative, and retraining improves performance to an AUROC of 0.751. The study reveals that listener conditioning helps only for listeners seen during training, that speaker identity is entangled with useful cues, and that backchannel rates vary significantly across individuals, prompting the authors to report both AUROC and event‑F1 metrics. whyItMatters":"The benchmark and evaluation tools provide a standardized, person‑disjoint testbed for advancing multi‑party backchannel prediction research."

By Mohammed Hafsati, Ahmed Loughzali
arXiv AI
Oct 2

False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift

The paper examines safety routers—systems that route user requests to different language models—and finds that their performance degrades significantly when evaluated under distribution shift. In standard benchmarks, routers appear effective because the best single model is chosen from the same evaluation data, but when the data distribution changes, the routing advantage diminishes or disappears. The study quantifies this bias across multiple safety corpora, showing that routers offer little benefit under realistic shift conditions and that recognition‑based defenses can be undermined by attackers who know the model being used.

By Amit Singh Bhatti, Vishal Vaddina
arXiv AI
Oct 2

Chaining Skills to Hijack LLM Agents

The paper "Chaining Skills to Hijack LLM Agents" introduces APEX, a method that constructs adversarial skill chains to manipulate large language model agents into performing attacker-selected actions. By exploiting the sequential invocation of open-source skills, APEX creates a record of genuine task progress that falsely signals user approval, enabling downstream skills to carry out malicious actions. Experiments on SkillsBench and GPT‑5.4 show high success rates, and a prompting defense reduces but does not eliminate the threat.

By Tian Dong, Zixuan Ma, Haodong Zhao, Huaien Zhang, Shaofeng Li, Hao Chen
arXiv AI
Oct 2

Removing spurious minima for planar features by skip connections

The paper investigates the loss landscape of shallow, bias‑free ReLU networks in a teacher–student setting, focusing on Gaussian population loss. It shows that adding a learned linear skip connection eliminates all spurious local minima with non‑negative student output weights when the student network is at least as wide as the teacher, whereas without the skip such minima persist even under overparameterization. Additionally, the study proves that the student network always learns the teacher’s feature subspace and that, in two dimensions, the effective width of an overparameterized student is bounded by twice the teacher’s width.

By Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin, Georg Loho
arXiv AI
Oct 2

Token Communication-Assisted Collaborative Embodied Artificial Intelligence: Concepts, Framework, and Opportunities

The paper introduces Token Communication (TokCom) as a native interface for Collaborative Embodied Artificial Intelligence (CEAI), where tokens act as compact semantic carriers and inference units for generative foundation models. It outlines a TokCom-assisted CEAI framework featuring a task‑adaptive communication protocol that includes a compact codebook, syntax rules, and contextual examples to guide token distillation and reconstruction over wireless channels. A case study on collaborative object transport shows that TokCom significantly reduces source payload bit consumption while maintaining task efficiency and robustness to noisy channels.

By Peng Yi, Ying-Chang Liang
arXiv AI
Oct 2

Detecting Inconsistencies in Model Specifications with LLM-as-Verifier Reasoning

Detecting Inconsistencies in Model Specifications with LLM-as-Verifier Reasoning introduces VeriSpec, a method that audits natural‑language model specifications directly using an LLM as a verifier. VeriSpec extracts structured rules, builds a topic‑guided graph to cluster related rules, and applies LLM‑based reasoning to find inconsistencies. In a case study on the OpenAI Model Spec, it identified 405 rules, validated five inconsistencies, and outperformed five baselines with the highest precision and lowest cost per validated inconsistency.

By Zichen Xie, Mrigank Pawagi, Lize Shao, Yang Hu, Wenxi Wang
arXiv AI
Oct 2

From Network Intrusion Detection to Blockchain-Backed Endpoint Detection and Response: Mapping the Landscape of Decentralized Detection-and-Response Architectures

This paper reviews blockchain-assisted intrusion detection and prevention systems for IoT and IIoT, noting that prior studies ignore the shift toward modern Endpoint Detection and Response (EDR) and Extended Detection and Response (XDR) architectures and lump blockchain’s roles together. It proposes a three‑axis taxonomy—detection‑system class, blockchain functional role, and response‑automation maturity—to classify recent research (2019‑2026). The authors identify gaps such as the lack of per‑endpoint blockchain‑anchored response loops due to latency, deployment, and community mismatches, and highlight cross‑cutting challenges like consensus latency, post‑quantum cryptography, smart‑contract vulnerabilities, and LLM‑based detection risks, concluding with a research agenda for hybrid on‑chain/off‑chain orchestration.

By Yahya Shahsavari, Sara Rouhani, Kaiwen Zhang
arXiv AI
Oct 2

Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis

The paper presents an analysis of asynchronous reinforcement learning for large language model post‑training, focusing on the impact of stale rollouts. It derives a convergence bound for GRPO‑style algorithms that balances the gradient estimator’s second moment and bias, and introduces a new group‑mass capping GRPO (GMC‑GRPO) method that reduces delay‑dependent bias. Experiments on Qwen3 models and reasoning benchmarks show that GMC‑GRPO outperforms existing baselines in robustness to large rollout delays.

By Qijia He, Ruinan Jin, Jun Luo, Shaofeng Zou, Yingbin Liang
arXiv AI
Oct 2

Cross-Lingual Alignment for Decoder-Only Models using MoE Routers

The paper introduces a new method for aligning cross‑lingual representations in decoder‑only large language models by leveraging the outputs of mixture‑of‑experts (MoE) routers instead of traditional hidden‑state alignment losses. This approach pools router outputs over many tokens, providing more reliable sequence‑level comparisons across languages. Experiments on four open‑source MoEs show that the routing loss not only aligns hidden representations but also improves multilingual performance across a diverse evaluation suite.

By Lucas Bandarkar, Clark Peng, Ahmed Haj Ahmed, Aditi Khandelwal, Nanyun Peng
arXiv AI
Oct 2

Iterative Policy Refinement through Semantic Rollout Analysis

The paper introduces a closed‑loop framework that refines structured policies for imitation learning by using large language models (LLMs) to analyze policy rollouts logged as semantically meaningful tabular data. By prompting the LLM to generate diagnostic code, the method identifies and corrects suboptimalities in the policy structure without human instruction. Experiments on car racing and door opening tasks show up to 15% performance improvement over zero‑shot LLM‑generated structures and a 75% reduction in compute needed to reach the same reinforcement learning performance.

By Feiyu Gavin Zhu, Qi Xu, Zhifei Deng, Zhigang Hua, Luke Simon, Jean Oh, Reid Simmons