AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computation and Language
Sep 25

Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts

The paper introduces ECHO, a paired diagnostic benchmark for evaluating Chinese real‑time spoken dialogue systems on turn‑taking. ECHO pairs examples that share the same overlap transcript but differ in preceding multi‑turn context, requiring either Yield or Keep actions, and also includes off‑talk cases to test unnecessary yielding. Experiments on four speech systems reveal that three systems over‑yield, correctly keeping the floor on fewer than 13% of backchannels, while the fourth system shows a more balanced performance, illustrating that interruption‑only evaluation can overestimate turn‑taking reliability.

By Shuofeng Zhao, Hongwei Cai, Wenke Fan, Qingxiang Guo, Zhou Wang, Dawei Yang, Zhiyang Zhou, Yingxin Shang, Weixu Wang, Lin Yang, Shuran Zhou, Yang Song
arXiv Computer Vision
Sep 25

FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining

FluidRain is a lightweight video deraining model that leverages a divergence‑free rain flow field to guide Loop‑in‑Loop attention across scales and neighboring frames, eliminating the need for explicit motion alignment. By projecting estimated rain‑flow onto a divergence‑free subspace, the method steers window attention along rain streaks, enabling efficient temporal aggregation with only 0.80 M parameters. Experiments on four benchmarks demonstrate competitive performance against larger models, and the authors introduce a new RainSyn‑Gust dataset and a physics‑based no‑reference metric for evaluating real‑rain removal.

By Pu Wang, Yongcong Wang, Wenhao Li, Xiang Chen, Guangwei Gao, Jinshan Pan, Siyuan Yao, Shujun Fu, Zhuoran Zheng
arXiv Computer Vision
Sep 25

An Automated Georeferencing Technique for Multi-Temporal Stope Point Clouds for Downstream Geotechnical Analysis

The paper introduces 3D-TARGeT, an automated method that uses inexpensive rectangular tags to register and georeference multi‑temporal UAV laser scans of underground mine stopes. By combining tag detection, geometric matching, and rigid transformation estimation, the technique achieves centimetre‑level accuracy, outperforming conventional automatic registration methods. The authors validated the approach on four simulated stope scans, demonstrating its potential to streamline data integration for geotechnical analysis and mine planning.

By Dibyayan Patra, Simit Raval, Pasindu Ranasinghe, Bikram Banerjee, Ismet Canbulat
arXiv Computer Vision
Sep 25

FounRef: Robust, Structure-Preserving, and Fast Metric Refinement of Frozen Monocular Foundation Priors with Sparse Anchors

FounRef is a training‑free method that refines frozen monocular foundation priors into dense metric depth by aligning them with sparse metric anchors. It validates anchors against the prior’s predictions, rejects misaligned ones, and applies a structure‑preserving solver to correct depth globally and locally while preserving fine geometry. The approach works out of the box on unseen cameras and scenes, achieving up to 24% lower depth error, 92% lower surface‑normal noise, and nearly 15× faster inference than a leading depth‑completion network.

By Dan Halperin, Mirko M\"ahlisch
arXiv Computer Vision
Sep 25

ComplexSync: High-Fidelity and Real-Time Lip Sync in Complex Scenarios

ComplexSync is a diffusion-based framework that delivers real‑time, high‑fidelity lip synchronization even in complex scenarios. It uses a dual‑stream joint training strategy to prevent reference‑frame leakage, a distillation‑based acceleration for single‑step denoising that reaches over 70 FPS, and a relational alignment loss that incorporates structural priors from Vision Foundation Models to improve robustness. The authors also introduce the first benchmark for complex lip synchronization, featuring more than 200 challenging video sequences and specialized metrics, and show that ComplexSync outperforms existing methods on both standard and complex tasks.

By Jiaran Cai, Xingpei Ma, Shenneng Huang
arXiv Computer Vision
Sep 25

When Misalignment Becomes Supervision: Structured Label Noise in Supervised Synthetic CT Generation

The paper examines how residual misalignments from registration procedures introduce structured label noise in supervised synthetic CT (sCT) generation. It shows that voxel‑wise metrics are heavily influenced by the consistency between training and evaluation registrations, and that training with anatomically consistent registrations reduces variability and improves robustness. Introducing a perceptual loss based on a pretrained Segment Anything encoder yields sharper, more anatomically coherent sCT and highlights the need for anatomy‑oriented evaluation.

By Valentin Boussot, Cedric Hemon, Caroline Lafond, Jean-Claude Nunes, Jean-Louis Dillenseger
arXiv Computer Vision
Sep 25

PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answering

PROVE is a black‑box hallucination detector for medical visual question answering that tailors its verification strategy to each question’s evidential structure. It classifies questions into three regimes, activates a subset of five operators per regime, and calibrates operator importance using deterministic question‑answer features to produce a risk score. On 8048 test samples across three medical VQA benchmarks and four state‑of‑the‑art vision‑language models, PROVE achieves an AUROC of 0.821, surpassing the best baseline by 0.159 with consistent improvements across all models and datasets.

By Keyang Zhou, Siyi Li, Zhongnan Shi, Qichao Ying, Wei Tang, Zhenxing Qian
arXiv AI
Sep 25

NNV3: Expanding Neural Network Verification to New Architectures and Domains

NNV3 is the latest version of the Neural Network Verification tool, a MATLAB framework for formally verifying deep learning models and learning‑enabled cyber‑physical systems. It builds on earlier NNV releases by adding new Star‑set members—ModelStar for weight perturbation, VolumeStar for video and 3D volumetric inputs, and GraphStar for graph neural networks—alongside a probabilistic reachability mode and FairNNV for fairness certification. The update also introduces benchmarks in malware detection, power‑system modeling, medical imaging, variable‑length time series, and action recognition, and provides unified documentation and tutorials.

By Anne M. Tumlin, Samuel Sasaki, Ben Wooding, Diego Manzanas Lopez, Muhammad Usama Zubair, Navid Hashemi, Hongchao Zhang, Waseem Abbas, Ipek Oguz, Meiyi Ma, Taylor T. Johnson
arXiv AI
Sep 25

Auditability Is Not One Property: Rule Overlap, Behavioural Agreement, and Composition in Reinforcement Learning

The paper introduces a protocol for auditing and composing reinforcement‑learning policies using discrete behavioral rules, defining auditability through six testable predicates such as trace integrity and rule coverage. Experiments show that overlapping rule sets do not guarantee behavioral agreement, and that rule‑based fusion often fails to outperform value‑based composition, highlighting limitations in current description layers. The authors provide an evidence‑bounded audit framework and outline future directions for more robust skill composition.

By Liu Hung Ming
arXiv AI
Sep 25

Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

The paper introduces Domain Recentering with Confidence Calibration (DRC), a training‑free technique that adapts CLIP to unlabeled target images by fitting a Gaussian mixture and subtracting a posterior‑weighted average of component means from each embedding. It further corrects residual class bias using a log‑prior adjustment based on confidence‑weighted predictions. DRC outperforms other methods, raising average accuracy on cross‑domain datasets by 4.13 and 5.07 points over zero‑shot CLIP for ViT‑B/16 and ResNet‑50, and maintains gains under ImageNet distribution shifts.

By Youngeun Seol, Jimin Shin, Heeseo Yoon, Uiwon Hwang
arXiv AI
Sep 25

Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs

The paper investigates a new attack method called output‑prefix attacks on reasoning LLMs, where an attacker prepends a malicious text to the model’s output, thereby conditioning all subsequent tokens on that prefix. The study systematically isolates the scratchpad reasoning channel as a vulnerable vector and compares three attack types—reasoning‑only, output‑prefix‑only, and combined reasoning‑plus‑output‑prefix—across both exposed and hidden reasoning models. Experiments on three 2026‑era frontier models (Gemini 3 Flash Preview, DeepSeek V4 Flash, and Claude Haiku 4.5) show that reasoning alone is largely ineffective, but adding a trivial output prefix can raise attack success rates to as high as 99% for some models, with contextual prefixes outperforming static ones and susceptibility varying by model.

By Luk\'a\v{s} Br\r{u}na, Robert Bridges, Adam Ek
arXiv AI
Sep 25

Structured Pose-Conditioned Flow Matching for Generative 5G CSI Augmentation

The paper introduces StructFlow-HPR, a structured pose‑conditioned flow matching framework that generates realistic 5G channel state information (CSI) paired with human pose data. It learns a continuous latent transport process from Gaussian noise to real CSI, preserving the receiver‑frequency topology via a reconstruction‑preserving autoencoder. A pose‑conditioned Transformer models the latent velocity field, enabling pose‑aligned CSI generation through ordinary differential equation sampling, which improves human pose recognition performance when data are scarce.

By Haojin Li, Anbang Zhang, Wai Ho Mow, Chenyuan Feng, Chen Sun, Haijun Zhang
arXiv AI
Sep 25

GHOST-Q: Towards Studying Grounding Hallucinations Overlooked Under Same-score TradeOffs in Quantized VLMS

GHOST-Q evaluates how post‑training quantization affects visual grounding in vision‑language models. The study compares three 8B VLM families across FP16, INT8, and NF4 precisions, pairing predictions to measure how compression redistributes grounding successes and failures. While most quantized variants maintain overall accuracy, several exhibit significant changes in hallucination‑sensitive conditions, and memory savings do not always translate to lower latency.

By Saim Rehman, Muhammad Shafique
arXiv AI
Sep 25

Unraveling the cognitive patterns of Large Language Models through module communities

The paper presents a network-based framework that links cognitive skills, LLM architectures, and datasets to uncover the cognitive patterns of Large Language Models. It identifies module communities within LLMs that exhibit skill distributions partially mirroring distributed cognitive organization seen in avian and small mammalian brains, yet distinct from the focalized specialization of specific biological systems. The study highlights that LLM skill acquisition benefits from dynamic, cross-regional interactions and neural plasticity, suggesting fine-tuning should leverage distributed learning dynamics rather than rigid modular interventions.

By Kushal Raj Bhandari, Pin-Yu Chen, Jianxi Gao
arXiv Statistics ML
Sep 25

Shrinking-Tube Concentration for Adaptive Markovian Stochastic Approximation

The paper establishes a shrinking‑tube concentration bound for projected stochastic approximation driven by an adaptive Markov chain, guaranteeing that after a chosen time every iterate stays within a tolerance that tightens over time. The bound’s probability of any exit after that time decays polynomially, and a matching lower bound shows this exponent is optimal under finite second moments. Extensions to recursions with martingale‑difference noise and predictable bias reveal how noise scale and bias affect exit‑probability decay and tube shrinkage, with applications to inventory learning and numerical gradient accuracy.

By Jin Li, Ye Luo, Xiaowei Zhang
arXiv AI
Sep 25

Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races

The paper investigates how tool‑using language‑model agents can safely commit changes to infrastructure when external state may change between read and commit. By distinguishing invalidating races from predicate‑preserving and irrelevant ones, the authors evaluate three commit‑time guard granularities—global epoch, read‑set version, and semantic commit predicate—using a deterministic simulator and three quantized model families. The study finds that only the complete predicate guard consistently eliminates unsafe commits, while freshness‑based guards block a large proportion of benign races and model‑side signals fail to replace precise semantic enforcement.

By Zihao Zheng, Jiayu Long, Baichuan Li, Junyi Yao
arXiv Machine Learning
Sep 25

Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration

The paper compares two popular data‑integration techniques—Stack‑SVD, which concatenates datasets before performing singular value decomposition, and SVD‑Stack, which first decomposes each dataset separately and then aggregates the leading singular vectors. By deriving exact asymptotic performance expressions and phase transitions in a proportional regime, the authors show that neither method uniformly dominates the other when unweighted, but optimally weighted Stack‑SVD outperforms optimally weighted SVD‑Stack when the low‑rank signal is fully shared. They also demonstrate that SVD‑Stack can excel with partially shared components and provide practical algorithms for estimating optimal weights, supported by simulations and genomic experiments.

By Tavor Z. Baharav, Phillip B. Nicol, Rafael A. Irizarry, Rong Ma
arXiv AI
Sep 25

WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks

WebArxiv is a reproducible benchmark designed to evaluate multimodal web agents on arXiv-related tasks. It consists of 510 static, time‑invariant tasks that require multi‑constraint paper retrieval, fine‑grained content extraction, and cross‑paper comparison, each with a deterministic ground truth. The benchmark highlights challenges for foundation‑model agents, such as over‑reliance on fixed interaction histories, and introduces a lightweight dynamic‑memory mechanism to improve adaptive retrieval and reasoning.

By Zihao Sun, Zijing Shi, Ling Chen
arXiv AI
Sep 25

PartHackBench: Certified Equal-Progress Stress Tests for Partial-Credit Tool-Agent Evaluation

PartHackBench is a new methodology for evaluating partial-credit tool agents that eliminates confounding factors by using a private certifier to ensure that compared trajectories match in both current-state predicate satisfaction and standardized agent attribution. The approach measures score inflation only after certification, and in experiments on 18 held-out tasks it produced matched adversaries for 15 tasks, revealing mean inflation of 0.252 and detecting no strict rollbacks. The study shows that semantic LLM judges are vulnerable to evaluator-targeted attacks, whereas PB-CSTE current-state controls guarantee zero inflation by construction.

By Hongye Yang, Zhihao Xie, Shengjun Xiong
arXiv AI
Sep 25

M$^2$PFN: End-to-End Disentangled Alignment for Generalizable Multimodal In-Context Learning in Alzheimer's Disease

M$^2$PFN is an end‑to‑end multimodal framework that extends the TabPFN in‑context learning engine to Alzheimer’s disease diagnosis by aligning 3D‑MRI and tabular features in a shared subspace. It performs differentiable inference through TabPFN’s transformer, back‑propagates gradients into the encoders, and incorporates a frozen tabular‑only prediction via a gated shortcut. On the ADNI cohort it achieves 65.55 % macro‑F1 and 82.21 % macro‑AUC, surpassing unimodal and multimodal baselines, and it generalizes to external cohorts without retraining.

By Lujia Zhong, Shuo Huang, Jianwei Zhang, Xinyu Nie, Yonggang Shi