AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Oct 8

One-Slide Calibration of Pathology Foundation Models

The paper introduces SlideRuler, a method that uses internal controls within a pathology slide to estimate and correct scanner‑induced shifts in tissue representation. By learning a transfer map from paired rescans, SlideRuler can calibrate a frozen foundation model from a single scan, reducing embedding distance by 16.3‑38.5% across two encoders and five SCORPION scanners. A source‑anchored variant further reduces source‑feature displacement while preserving most alignment gains, demonstrating consistent calibration across imaging systems.

By Ming Ren Hou, Tianyi Huang
arXiv Machine Learning
Oct 8

DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters

DiTS introduces a Multimodal Diffusion Transformer designed for time‑series forecasting that incorporates heterogeneous exogenous covariates. The model treats endogenous targets and exogenous covariates as separate modalities, jointly conditioning future generation on target history and available covariates. By employing flow matching and Time‑aligned Modulation, DiTS achieves strong deterministic and probabilistic forecasting performance across diverse covariate‑aware tasks.

By Haoran Zhang, Haixuan Liu, Yong Liu, Yunzhong Qiu, Yuxuan Wang, Jianmin Wang, Mingsheng Long
arXiv Computer Vision
Oct 8

Do Generative Priors Align with Human Naturalness Perception?

The study investigates whether the priors learned by visual generative models align with human perceptions of image naturalness. By measuring prediction errors across 25 image and video generators and using paired relational interventions that alter facial configurations or illumination while preserving low-level statistics, the authors find that loss differences replicate human-like sensitivities, including the Thatcher effect and shape-dependent illumination responses. These loss differences correlate strongly with continuous human naturalness judgments (up to r = 0.84 for faces) and remain informative after controlling for feature distances and standard image quality metrics, revealing that human-like naturalness judgments can dissociate from generic violation detection.

By Taiki Fukiage
arXiv Machine Learning
Oct 8

The Deceptive Bandit Problem: Exploratory Coupling and the Fragility of Multi-Agent Learning

The paper investigates how randomized exploration in bandit learning can be compromised when an adversarial agent gains access to another agent’s exploratory signals. In a two‑player strongly monotone setting, the deceptive agent couples its actions with the leaked information to steer the learning dynamics toward a new steady state, termed the deceptive Nash equilibrium (DNE). The authors prove that the deceptive bandit learning dynamics converge to an arbitrarily small neighborhood of the DNE while maintaining optimal convergence rates, even under relaxed smoothness assumptions, and illustrate the impact on costs in a resource‑allocation game.

By Michael Tang, Mahmoud Abdelgalil, Jorge I. Poveda
arXiv Machine Learning
Oct 8

Temporally Interpretable Differentiable Decision Trees

The paper introduces temporally interpretable differentiable decision trees (DDTs) that incorporate action chunking to better align tree-based decision making with multi‑timestep planning. Two new policy‑gradient algorithms and an information‑theoretic tree‑restructuring method are proposed to enable efficient training of these trees. Experiments on four simulation environments show that warm‑starting action‑chunked DDTs from a distilled policy yields trees that match neural network performance in three domains while using up to 80% fewer parameters.

By Eisuke Hirota, Aarav Sane, Rohan Paleja
arXiv Machine Learning
Oct 8

DenoFlow: Flow Matching for SSVEP Denoising under Real Physiological Artifacts

DenoFlow introduces a flow‑matching approach to denoise steady‑state visual evoked potential (SSVEP) EEG signals, avoiding direct mapping from noisy to clean data. It uses a field network that predicts the velocity of a straight path between contaminated and clean trials, integrating this field to produce denoised outputs while simultaneously training a classifier to supervise the result. Experiments on two public SSVEP datasets demonstrate that DenoFlow surpasses seven baseline denoising models in both signal fidelity and downstream decoding accuracy.

By Zhentao He, Ziwei Wang, Dongrui Wu
arXiv Machine Learning
Oct 8

Adjoint-Based Calibration and Optimal Control of Stochastic Multiscale Bioprocess Digital Twins

The paper presents a bias-aware digital‑twin calibration and control framework for multiscale bioprocess models, using stochastic differential equations and quasi‑likelihood estimation with adjoint sensitivity analysis. It introduces SDE generator‑based moment expansions to quantify truncation‑induced parameter bias and forward‑backward adjoints to trace how calibration uncertainty affects value functions and policy performance. An Actor‑Simulator algorithm is proposed to jointly update model parameters, select informative experiments, and optimize control policies, achieving improved calibration accuracy, sample efficiency, and control performance in numerical studies.

By Keilung Choy, Wei Xie
arXiv Machine Learning
Oct 8

Estimating Model-Level Membership Inference Vulnerability Without Reference Models

The paper introduces a method to estimate a model’s vulnerability to the Likelihood Ratio Attack (LiRA) without training reference models, using only the target model’s train and test loss distributions. It shows that LiRA’s per‑sample signal can be decomposed into a variance‑ratio term and a residual mean‑shift term, and that different loss‑distribution shapes dictate which reference‑free proxy to use. Two proxies are presented: the LOSS attack TNR for heavy‑tailed losses and the LOSS attack AUC for symmetric losses, both achieving low RMSE in predicting LiRA TPR across multiple architectures and datasets.

By Euodia Dodd, Nata\v{s}a Kr\v{c}o, Igor Shilov, Matthew Wicker, Yves-Alexandre de Montjoye
arXiv Machine Learning
Oct 8

Routing-Aware Safety Alignment for Mixture-of-Experts Models

The paper introduces RASA, a routing‑aware safety alignment framework for Mixture‑of‑Experts (MoE) language models. RASA repairs Safety‑Critical Experts by selectively fine‑tuning only those experts that are disproportionately activated during jailbreak attacks, while keeping routing fixed and enforcing routing consistency with safety‑aligned contexts. Experiments on two MoE architectures show that RASA achieves near‑perfect robustness against diverse jailbreak attacks, generalizes across attacks, reduces over‑refusal, and maintains performance on standard benchmarks such as MMLU, GSM8K, and TruthfulQA.

By Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang
arXiv Machine Learning
Oct 8

Best Arm Identification for Bandits with Shifting Means

arXiv:2610. 10488v1 Announce Type: cross Abstract: We study the best arm identification problem in a stochastic environment with a novel form of adversarial perturbations, which we coin Shifting Means.

By Lukas Zierahn, Wouter M. Koolen, Shubhada Agrawal, Christina Katsimerou, Dirk van der Hoeven
arXiv Computation and Language
Oct 8

WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection

WaveScat introduces a new family of feature extractors for speech deepfake detection that merges wavelet scattering transform (WST) with self‑supervised learning. By cascading wavelet convolutions with modulus nonlinearities, WaveScat produces deformation‑stable, multi‑scale features that capture both low‑level spectral details and higher‑level information. Experiments on Deepfake‑Eval‑2024 and cross‑dataset tests (SpoofCeleb, In‑the‑Wild, ASVspoof 5) demonstrate that WaveScat significantly outperforms existing front‑ends, with a small averaging scale and high‑frequency directional resolution proving crucial for detecting subtle artifacts.

By Xi Xuan, Davide Carbone, Wenxin Zhang, Tomi H. Kinnunen
arXiv Machine Learning
Oct 8

Learning Cross-Model Activation Alignments with Explicit Many-to-Many Layer Maps

The paper introduces MATCHA, a method that learns explicit many-to-many layer maps between independently trained language models. MATCHA decomposes the alignment into a layer map—an explicit target‑by‑source matrix—and a shared feature map, allowing joint learning of layer correspondence and feature transformation from prompts. Experiments on 42 model pairs across three families show that MATCHA reconstructs target activations more faithfully and improves retrieval metrics, while enabling transfer of activation‑space interventions between models.

By Alina Sudakov, Guy Bar-Shalom, Fabrizio Frasca, Haggai Maron
arXiv Machine Learning
Oct 8

HydroSphere: A Framework for Governed, Self-Healing Wastewater Infrastructure

HydroSphere is a governed, data‑driven framework that enables real‑time monitoring, forecasting, treatment optimization, and fault recovery for wastewater infrastructure. It combines a hybrid TCN‑LSTM model for multi‑step water‑quality forecasting, a PPO‑based adaptive dosage optimization module, and a SHADE autoencoder for anomaly detection, all evaluated on 2.82 million measurements from 1940 to 2023. The system incorporates safety bounds, human oversight, and tiered governance to promote safer, more adaptive water management aligned with UN Sustainable Development Goals 6 and 13.

By Prabu, Fancy C, Suresh A, Srini Ramaswamy
arXiv Machine Learning
Oct 8

ORACLE: Optimizer-Relative Alignment for Constrained LEarning

The paper introduces ORACLE, a method for constrained learning that evaluates constraint compatibility after the optimizer has taken a step, rather than before. ORACLE performs a joint endpoint linearization of diverse constraint families, constructs an alignment in the optimizer’s geometry, bounds its authority, and commits the adjustment only after validation. Experiments on eight PDE benchmarks with four different optimizers show that ORACLE matches or improves performance in 94% of configurations, outperforming other constraint-handling approaches.

By Utkarsh Grover, Wyatt Mackey, Kaixun Hua, J. Morris Chang, Xiaomin Lin
arXiv Machine Learning
Oct 8

Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata

The paper introduces spatial induction heads, a two‑layer gather‑and‑match mechanism that enables transformers to perform in‑context learning on multidimensional cellular automata. By reconstructing local spatial neighborhoods and matching them against prior occurrences, the method achieves Bayesian counting and can approximate the Bayesian posterior for stochastic rules or predict deterministically for deterministic rules. Experiments show that trained transformers generalize to unseen rules in one and two dimensions, achieving near‑perfect rollouts and KL divergences below 0.005 nats from the Bayes‑optimal predictor, with attention patterns supporting the proposed computation.

By Kimia Kazemian, Menghan Xu, John Thickstun, Sarah Dean
arXiv Machine Learning
Oct 8

An Informational Curse of Horizon in Goal-Conditioned Policy Learning

The paper identifies an informational curse of horizon in goal-conditioned policy learning, showing that longer goal relabeling horizons during training can hurt policy generalization and performance. Controlled experiments with oracle planners separate training horizons from test horizons, revealing that behavioral cloning policies degrade with longer horizons while reinforcement learning mitigates this effect. The authors attribute this to reduced conditional mutual information between actions and hindsight-relabeled goals, and demonstrate that distilling short-horizon policy Jacobians into long-horizon policies improves performance, especially in combinatorial manipulation tasks.

By John L. Zhou, Yuxuan Dong, Jonathan C. Kao
arXiv Computation and Language
Oct 8

TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning

The paper introduces TOPOGRAPHRAG-BENCH, a benchmark for evaluating multimodal Graph Retrieval-Augmented Generation (GraphRAG) systems on layout‑grounded evidence reasoning. It contains 2,024 questions derived from 201 long, visually rich documents, designed around three controlled topologies—single‑hop retrieval, bridge‑chain reasoning, and multi‑source synthesis—while employing counterfactual validation to ensure question integrity. Experiments show that multimodal GraphRAG outperforms text‑only and page‑level visual retrieval models, yet still struggles with incomplete visual‑textual alignment and multi‑unit composition, highlighting the need for GraphRAG systems that explicitly model document layouts and cross‑modal evidence alignment.

By Ruochi Li, Jianzhe Lin, Haoxuan Zhang, Haihua Chen, Junhua Ding, Edward Gehringer, Yang Zhang
arXiv Machine Learning
Oct 8

Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling

The paper introduces a scalable Bayesian method for conditional density estimation using a logistic Gaussian process. It models the log conditional density with a separable covariance— a Matérn kernel on the response dimension and a Nyström-feature kernel on covariates— and samples the latent field via kinetic Langevin dynamics instead of traditional approximations. Experiments on photometric‑redshift data with up to 3.9 million observations show the approach matches state‑of‑the‑art tabular foundation models on density and calibration metrics.

By Daniel Paulin, \'Ad\'am Jung, Andr\'as A. Bencz\'ur