AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 7

LLM-Assisted Behavioural and Scenario Augmentation for Agent-Based Energy Adoption Models

The paper introduces a hybrid framework that uses large language models (LLMs) to assist in designing behavioural and scenario specifications for an agent‑based model of solar photovoltaic adoption by Irish dairy farms. It integrates bounded behavioural rubrics—conservative, balanced, and optimistic—with structured scenario specifications into a calibrated ABM, preserving the original techno‑economic adoption mechanism while adding controlled behavioural modulation and scenario‑driven uncertainty analysis. Experiments across various policy settings and Monte Carlo simulations show stable, economically plausible outcomes, with up to a 13% increase in behavioural adoption compared to a logistic baseline, without causing unrealistic saturation dynamics.

By Iias Faiud, Hossein Khaleghy, Michael Schukat, Karl Mason
arXiv AI
Sep 7

MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning

MePo++ is a post‑training framework designed for general continual learning (GCL) that unifies representation refinement and reconciliation. It introduces MetaPrep, which enhances representation plasticity via unsupervised meta‑refinement on pseudo continual sequences, and StreamAlign, which maintains stability by reconciling online features with a stable pretrained geometry. Experiments across various pretrained models, datasets, and continual learning baselines show that MePo++ consistently improves performance in PTM‑based GCL.

By Guanglong Sun, Kanglei Zhou, Liyuan Wang, Qi Cheng, Hongwei Yan, Shuang Cui, Hang Su, Jun Zhu, Yi Zhong
arXiv AI
Sep 7

ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding

ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding introduces a model‑agnostic framework that adaptively aligns EEG signals with visual semantics. It replaces fixed visual or textual anchors with EEG‑aware class‑level contrastive supervision and employs structure‑consistent interpolation to preserve channel‑wise and temporal importance. Across multiple evaluation settings—including subject‑dependent, subject‑independent, strict cross‑subject transfer, and continual adaptation—ProCA delivers significant performance gains, achieving relative Top‑1 improvements ranging from 7.4% to 28.1%.

By Kanglei Zhou, Chunyan Lan, Dongyang Li, Jun Zhu, Liyuan Wang
arXiv AI
Sep 7

Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?

The paper proposes a new method for evaluating AI accountability by analyzing the structural quality of a model’s defense for its decisions, using a four‑phase dialectical protocol based on Walton’s argumentation schemes and Govier’s criteria. Applied to nine large language models and 200 ambiguous moral-choice items, the study finds that models generally defend their reasoning well above the rubric minimum, though failures cluster on grounds and sufficiency and correlate with epistemic hedging. The protocol also reveals that models often present different argument schemes in justification than in reasoning, detects indefensible defenses, and highlights challenges in assessing retraction in AI alignment.

By Daan R. Henselmans, Derck W. E. Prinzhorn, Arno Libert
arXiv AI
Sep 7

What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection

The paper investigates data efficiency and selection in On‑Policy Distillation (OPD) for large language models. It shows that 1‑shot OPD—training on a single example—consistently improves performance, especially when the example is hard, and that longer chain‑of‑thought (CoT) paths drive the gains rather than token entropy. Based on these findings, the authors propose a simple hard‑example selection strategy that, using only eight carefully chosen hard examples, matches the performance of a 17,000‑example baseline across models from 1.5B to 7B parameters.

By Zhinan Hou, Jiaqi Zhang, Xunliang Cai, Keyou You
arXiv AI
Sep 7

AI for Computational Design Science: A Responsible Human-AI Framework and Case Study on Short-Form Video Safety Surveillance

The paper introduces AI4CDS, a five‑phase framework that guides how AI can participate in computational design science while keeping researchers responsible for domain grounding, verification, and scientific judgment. It emphasizes principles such as graduated trust, reversibility, auditability, and differentiated reproducibility. The authors demonstrate the framework with ChildRiskGuard, an interpretable system that detects child‑inappropriate short‑form videos, achieving an F1 score of 0.769 and outperforming generic content‑safety models.

By Wenli Zhang, Jiaheng Xie, Zhihe Pan, Yidong Chai, Xiao Fang, Sudha Ram
arXiv AI
Sep 7

Commonsense Reasoning in Computer Vision: Foundations, Recent Advancements, and Future Directions

The paper surveys how commonsense reasoning is being integrated into computer vision, moving beyond traditional CNNs that only detect objects. It reviews methods that use knowledge graphs, scene graphs, neuro-symbolic models, and transformers to add contextual understanding, thereby improving object recognition and spatial reasoning. The authors also discuss current limitations such as dataset bias and knowledge gaps, and propose future research directions in cross‑modal reasoning, scalable knowledge injection, and hybrid architectures.

By Bahar Uddin Mahmud, Sumit Barua, Guan Yue Hong, Ajay Gupta, Hexu Liu
arXiv AI
Sep 7

A Deep Generative Model for Synthesizing Labeled Wireless Signals

The paper introduces Inter-Instance Generative Adversarial Networks (IIns‑GAN), a deep learning approach for synthesizing realistic labeled wireless signals. Unlike traditional environmental‑model based methods, IIns‑GAN adapts to various scenarios and produces signals that closely match the physical characteristics of real measurements. Experiments on public Ultra‑Wideband datasets show that the generated signals improve model training for tasks such as distance estimation and environment identification.

By Yuxiao Li, Keke Hu, Santiago Mazuelas, Yuan Shen
arXiv AI
Sep 7

Scalable Context Orchestration for Serving LLMs Over Voice

The paper introduces llmovoice, a middleware that explicitly models voice context for large language model (LLM) serving in voice AI applications. By incorporating speaking rate, background noise, packet loss, and other paralinguistic factors into a bounded context, llmovoice guides the LLM to generate more aligned responses. Experiments show significant reductions in speaking‑rate errors, false interruptions, and model usage costs, especially in long voice sessions.

By Linyi Jiang, Silvery D. Fu, Yifei Zhu
arXiv AI
Sep 7

Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning

The paper evaluates three open‑weight large language models—Gemma3‑12B (USA), Bielik‑11B‑v3 (Poland), and Qwen3‑4B (China)—against World Values Survey data for 63 demographic personas across three countries, using normalized Wasserstein distance to measure cultural misalignment. Surprisingly, none of the models shows a preference for its home country; Qwen3‑4B, built in China, has the highest misalignment for Chinese respondents. Targeted LoRA fine‑tuning on the five worst‑case personas, with fewer than 1,200 training pairs and under 15 minutes on a single GPU, reduces bias by 16.8% for Bielik‑11B, but the fine‑tuning redistributes bias rather than eliminating it, shifting worst‑case personas from American to Chinese elderly.

By Antoni Czolgowski, Abel Iyasele
arXiv AI
Sep 7

Building a research-software catalog with a coding agent: from hackathon prototype to public deployment

The paper reports on building a research-software catalog using a coding agent, starting from a three‑day hackathon prototype and moving to public deployment. It details the engineering work needed—adversarial review, data‑quality checks, browser validation, and publication safeguards—to ensure reliable operation, noting that silent failures were more problematic than crashes. The authors then examine applying these lessons to a larger, human‑curated portal (MateriApps) that combines curated metadata, external documentation, vector search, and local language‑model generation, finding that explicit validation, monitoring, and repeated review remain essential for AI‑assisted software portals.

By Kazuyoshi Yoshimi, Satoshi Terasaki, Gotai Yamada
arXiv AI
Sep 7

Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

The paper investigates how large language models balance helpfulness and safety by refusing harmful queries while responding to benign ones. It decomposes safety-tuning responses into a boilerplate refusal statement and a rationale, finding that the statement causes false refusals by relying on superficial cues. Training on rationales alone reduces false refusals without compromising safety performance, suggesting that fine‑grained safety supervision is essential for better alignment.

By Minji Kim, Hyounghun Kim
arXiv AI
Sep 7

PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation

PRISM‑Bench is an audio‑centric diagnostic benchmark for text‑to‑audio‑video generation, built from 900 human‑verified samples. It evaluates audio along two axes—audio type (speech, music, sound) and sound‑source visibility (on‑screen vs. off‑screen)—across four perceptual dimensions (audio‑visual coherence, audio quality, audio expressiveness, and prompt following) using 35 fine‑grained criteria. The benchmark employs an enhanced MLLM‑as‑a‑Judge protocol that aligns strongly with human raters, revealing a performance gap between frontier and open‑source T2AV models and highlighting overfitting to perceptual fidelity while struggling with complex grounding and control tasks, especially for music and synchronized on‑screen audio.

By Yuchen Sun, Qian Yang, Jun Wang, Detai Xin, Guoqiao Yu, Guanglu Wan, Qi Jia
arXiv AI
Sep 7

MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate

MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.

By Garvit Joshi (Graphic Era University, Dehradun, India), Stavya Dhyani (Graphic Era University, Dehradun, India), Jasmine (Graphic Era University, Dehradun, India), Arun Chauhan (Graphic Era University, Dehradun, India)
arXiv AI
Sep 7

Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program Repair

The paper investigates hallucination in large language model–based automated program repair (APR). It defines hallucination as producing patches or intermediate artifacts that are not grounded in available repair evidence, and analyzes it across final patches and intermediate tasks such as triggering test case identification, line coverage prediction, and additional test case generation. Experiments on 832 Defects4J bugs show that only 21.0%–55.9% of patches pass the developer test suite, with 72.7% of sampled repairs exhibiting hallucinations, often due to incorrect causal localization or repair strategies.

By Xuemeng Cai, Jiakun Liu, Linhan Yang, Wei Ma, Lingxiao Jiang
arXiv AI
Sep 7

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

MCPO introduces a two-stage method to compress multimodal chain-of-thought reasoning. The first stage uses a step-level Normalized Cross-Modal Mutual Information pruning algorithm to remove visual-independent steps, reducing redundancy and hallucinations. The second stage fine-tunes the model with an asymmetric multimodal length-controlled preference loss, achieving up to 69.5% shorter reasoning chains and a 3.34× inference speedup while keeping accuracy.

By Guangheng Yang, Zhenliang Ni, Zhenkai Wu, Han Shu, Juan Feng, Wenming Yang, Jie Hu
arXiv AI
Sep 7

Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

The paper explores using a language model’s low‑level symbolic skill—specifically SQL—to detect hallucinations without fine‑tuning. By having the model construct an SQL database from reference documents, it can reason over both the source and the model’s output, creating a neurosymbolic check. Experiments on RAGTruth and DiaHalu show this method outperforms direct prediction and rivals state‑of‑the‑art detectors, highlighting the value of leveraging inherent symbolic competences in LLMs.

By Renato Vukovic, Hsien-chin Lin, Carel van Niekerk, Benjamin Ruppik, Michael Heck, Shutong Feng, Nurul Lubis, Milica Gasic
arXiv AI
Sep 7

Qlippy: A Retrieval-Augmented GenAI Assistant for Reproducible Quantum Workflows and Experiment Tracking

Qlippy is a retrieval‑augmented generative AI assistant designed to support reproducible quantum software development. It is embedded in the development environment and grounds its responses in a curated corpus of quantum‑software‑engineering knowledge, explaining reproducibility and provenance concepts in context. The assistant augments Qiskit programs with MLflow‑based experiment tracking that follows the QProv schema, thereby reducing reliance on large language models and enabling low‑cost, privacy‑preserving local deployment.

By Mahee Gamage, Vlad Stirbu
arXiv AI
Sep 7

A Structured Debate-Mixture-of-Agents Framework for Complex Clinical Diagnostic Decision Support

The paper introduces Debate-Mixture-of-Agents (DMoA), a multi‑agent framework that structures role‑based interactions to mimic iterative diagnostic reasoning in clinical settings. Evaluated on 297 rare disease cases and 1,719 challenging cases, DMoA outperformed a GPT‑4o baseline, improving most likely diagnosis accuracy by 10.21 percentage points and safety rate by 11.36 percentage points. Ablation studies and further analyses revealed that these gains stem from the structured workflow rather than merely adding more models or longer outputs, and that performance benefits are influenced by the chosen structure, base model strength, and token budget.

By Chang Xia, Leilei Ouyang, Huimin Wang, Yong Zhao, Kang Li
arXiv AI
Sep 7

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

The paper introduces TIER, a Threat Implicitness Benchmark designed to evaluate large language model (LLM) safety behaviors across four risk domains and four threat levels, ranging from explicit harmful requests to sophisticated jailbreaks. Responses are scored on a six-label behavior scale by two independent LLM judges. Experiments on six open-weight LLMs reveal that safety behaviors change gradually with threat level, contextual prompts produce the most varied responses, and jailbreaks expose significant robustness gaps, underscoring the importance of behavior-aware safety evaluation.

By Thu-Hien Trinh-Thi, Hai-Yen Vong, Thanh-Ha Ung-Dung, Tram Ho