AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Oct 2

Multicalibration for Unbiased Model-Based Prevalence Estimation

The paper presents multicalibration as a method for unbiased prevalence estimation when measurement devices (diagnostic tests, classifiers, or large language models) exhibit covariate shift. It shows that standard calibration fails under such shift, while multicalibration—calibrating conditional on input features—maintains near-zero bias in simulations and real-world applications. The authors demonstrate the approach on employment data across U.S. states and political text classification across four countries, emphasizing the need for calibration data that covers key feature dimensions.

By Fridolin Linder, Thomas Leeper, Daniel Haimovich, Niek Tax, Lorenzo Perini, Milan Vojnovic
arXiv AI
Oct 2

FairSSL: Fair Multimodal Self-Supervised Learning

FairSSL is a multimodal self‑supervised learning framework that treats data heterogeneity as a fairness resource instead of a limitation. It replaces strict alignment with a subject‑aware Variance‑Invariance‑Covariance Regularization objective, enforcing alignment only across segments from the same subject. The method includes segment‑based pooling for variable‑length modalities and regularizes representations to promote within‑subject variability, cross‑modal and cross‑subject invariance, and decorrelation, theoretically bounding the score gap between protected groups and empirically outperforming baselines on heterogeneous multimodal datasets.

By Jiaee Cheong, Abtin Mogharabin, Paul Liang, Hatice Gunes, Sinan Kalkan
arXiv AI
Oct 2

UrbanVLA: A Vision-Language-Action Model for Urban Micromobility

UrbanVLA is a Vision‑Language‑Action framework designed to enable delivery robots to navigate large‑scale urban environments using long‑horizon route instructions. The model aligns noisy route waypoints with visual observations and plans trajectories, trained through a two‑stage pipeline of supervised fine‑tuning on simulated data and reinforcement fine‑tuning on mixed simulation and real‑world data. Experiments show UrbanVLA outperforms strong baselines by over 55% on the SocialNav task and demonstrates reliable real‑world navigation in large urban settings.

By Anqi Li, Zhiyong Wang, Jiazhao Zhang, Minghan Li, Yunpeng Qi, Zhibo Chen, Zhizheng Zhang, He Wang
arXiv AI
Oct 2

Constant-Time Planning for Chaining Collision-free Motion to Manipulation Behaviors

The paper introduces the Behavioral Constant-Time Motion Planner (B-CTMP), an extension of Constant-Time Motion Planning that handles two-step manipulation tasks in semi-structured environments. B-CTMP constructs neighborhoods in object-pose space and uses statistical certification to ensure a user-specified success rate, caching plans only when repeated rollouts meet this threshold. The method is evaluated on shelf picking, plug insertion, and wheel replacement, showing consistent success where baseline planners fail and rejecting infeasible poses in constant time.

By Nayesha Gandotra, Itamar Mishani, Lai Yuan, Oren Salzman, Maxim Likhachev
arXiv AI
Oct 2

Explainability of Complex AI Models with Correlation Impact Ratio

The paper introduces ExCIR, a new explainability metric that addresses limitations of existing post‑hoc explainers by accurately handling correlated features and avoiding costly perturbations. ExCIR is theoretically grounded, computationally efficient, and remains stable under noise and sampling variations. Experiments on EEG, synthetic vehicular data, Digits, and Cats‑Dogs datasets show that ExCIR provides more interpretable feature explanations than current methods, and its information‑theoretic extension supports multi‑output and class‑conditioned explanations at scale.

By Poushali Sengupta, Rabindra Khadka, Sabita Maharjan, Frank Eliassen, Yan Zhang, Shashi Raj Pandey, Pedro G. Lind, Anis Yazidi
arXiv AI
Oct 2

Pepti-drift: Scalable Safe-Active Peptide Generation Without Inference-Time Guidance

Pepti-drift is a one‑step peptide generation framework that uses a single latent refinement and parallel decoding, avoiding costly inference‑time guidance. It incorporates attraction toward target‑specific binders and repulsion from liability‑associated regions, achieving an 18.37% predicted Safe‑Active yield across 88 held‑out targets. The method delivers competitive gains to multi‑property‑guided baselines while reducing generation cost by 468×, enabling scalable and fair high‑throughput peptide design.

By Takashi Fujiwara, Hikaru Shindo, Kaushalya Madhawa, Jun Jin Choong, Shuan Chen, Yuna Oikawa, Yiming Zhang, Gyubok Lee, Keisuke Ozawa
arXiv AI
Oct 2

Mitigating LLM Over-Refusal via Dynamic Semantic Routing Calibration

The paper addresses the problem of large language models (LLMs) over-refusing to comply with benign safety-related instructions. It identifies that a small number of hypersensitive safety heads in transformer attention misfire on hard-safe prompts, causing abnormal attention entanglement and high-entropy routing conflicts that lead to refusals. The authors propose Semantic Routing Calibration (SRC), a lightweight, training‑free inference framework that dynamically suppresses these problematic heads and fuses logits to restore trustworthy reasoning while preserving intrinsic safety performance.

By Zixuan Wang, Bingjie Zhang, He Zhao, Dandan Guo
arXiv Machine Learning
Oct 2

Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIP

The study investigates whether text conditioning improves neural partial differential equation (PDE) surrogates by using OperatorCLIP. Experiments compare an unconditioned Fourier Neural Operator (FNO), a constant-sentence FiLM control, and a task‑specific text description trained with contrastive alignment across Darcy2D, ShallowWater2D, and 3D compressible Navier‑Stokes. Results show that constant conditioning yields lower mean test error on 2D tasks, while task text plus alignment performs similarly or worse, with significant seed uncertainty and no clear semantic ordering from prompt interventions.

By Aadi Dash, Lennon J. Shikhman, Michael Galarnyk
arXiv Machine Learning
Oct 2

LEARN-TS: LLM-Enhanced Alignment and Reconstruction with Normality Guidance for Multivariate Time-Series Anomaly Detection

LEARN-TS is a method for multivariate time‑series anomaly detection that uses a frozen language model to generate two semantic representations: one that captures window‑specific observation context without exact numerical values, and another that provides a fixed, dataset‑agnostic normality prompt. These representations guide a patch‑masked reconstruction process, allowing the model to estimate normality discrepancy and produce timestamp‑level anomaly evidence. Experiments on four benchmarks show that LEARN‑TS achieves the best mean performance in 13 of 16 dataset‑metric comparisons, and ablation studies confirm the benefits of observation conditioning, joint normality alignment, and semantic references.

By Jahyeob Koo, Kio Yun, Byoungmo Koo, Jun-Geol Baek
arXiv Machine Learning
Oct 2

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs

The paper introduces SparLeak, a side‑channel attack that exploits a new GPU micro‑architectural leakage called Sparsity‑Induced Memory Access (SIMA) caused by sparse attention in large language models. By capturing SIMA traces during LLM inference, SparLeak can infer user‑query attributes from prefill‑phase traces and reconstruct autoregressive responses from decoding‑phase traces. Experiments on three LLM architectures, three sparse attention mechanisms, and three privacy‑sensitive datasets show high success rates (90.9% for attribute inference and 87.3% for response reconstruction), underscoring the need to address SIMA leakage in sparse‑attention deployments.

By Fahao Chen, Linkang Du, Jinhao Zhou, Peng Li, Zhou Su
arXiv Machine Learning
Oct 2

Robust Risk-Sensitive Reinforcement Learning from Corrupted Human Feedback

The paper introduces WSP-CVaR-RLHF, a risk‑sensitive reinforcement learning algorithm that learns from human feedback while handling corrupted or adversarial preference labels. It uses uncertainty‑weighted reward estimation and optimistic CVaR planning to achieve a regret bound that separates clean statistical cost from corruption penalties. Experiments with four adversarial attacks show that the method consistently reduces cumulative regret compared to an unweighted robust baseline while maintaining confidence‑set coverage.

By Xinyi Ni, Lifeng Lai
arXiv Machine Learning
Oct 2

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

The paper investigates the relationship between representation and function in neural networks, using analytical two‑layer linear models and simulations of nonlinear networks. It demonstrates that functional similarity and representational similarity can be dissociated: networks may share representations without sharing functions, and vice versa. The study also finds that robustness to input noise or generalization error does not constrain representations, whereas robustness to parameter noise forces networks to adopt task‑specific representations, indicating that representational alignment reflects computational advantages beyond mere functional alignment.

By Lukas Braun, Erin Grant, Andrew M. Saxe
arXiv Machine Learning
Oct 2

Certification-Based Differentially Private Learning

The paper extends the abstract gradient training (AGT) framework to provide tighter differential privacy guarantees for both private prediction and private learning. It introduces Abstract Gradient Sampling (AGS) to analyze privacy in continuous, unbounded regression and offers theoretical and empirical evidence that these methods yield tighter bounds than global-sensitivity baselines, even in previously unbounded settings. The authors also demonstrate that their private learning algorithm can outperform standard private learners under comparable conditions.

By Mihnea Ghitu, Matthew Wicker
arXiv Machine Learning
Oct 2

VirusCascade: Hijacking Collaborative Reflection in LLM-Powered Recommender Agents

The paper introduces VirusCascade, a black‑box attack that exploits the collaborative reflection mechanism in LLM‑powered recommender agents. By leveraging reflective persistence and cross‑agent propagation, the attack promotes a target item through semantic rationalisation and structural positioning, achieving state‑of‑the‑art targeted exposure on four real‑world datasets. The study demonstrates that existing static‑pipeline attacks cannot exploit this recurrent, multi‑agent amplification pathway.

By Yurong Hao, Wen Zhou, Guowei Guan, Tiantong Wu, Fuyao Zhang, Wei Yang Bryan Lim
arXiv Machine Learning
Oct 2

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

The paper examines how public institutions, particularly a Canadian college, can address inequities in proprietary Early Warning Systems (EWS) when they cannot modify the underlying AI models. By testing six post‑hoc fairness interventions on a research EWS under realistic procurement constraints, the authors compare fairness, accuracy, and demographic disparities, and introduce error‑type profiling to track how interventions shift false positives and false negatives. The results show that while interventions can redistribute disparities, they do not consistently reduce them; some favor already‑advantaged groups, leaving small, marginalized groups poorly served, illustrating the limits of fairness work under vendor‑controlled systems.

By Kelly McConvey, Angelina Zhai, Rebecca Li, Shion Guha
arXiv Machine Learning
Oct 2

Inference-Layer Security: Defending Against Adversarial Inference and Infrastructure Abuse

The paper presents a study on securing large language model (LLM) inference services against adversarial use, such as jailbreaking, denial‑of‑service, and distillation attacks. It introduces a structural causal model to generate a realistic, labeled dataset of user sessions, including coordinated multi‑account campaigns and varying label observability. Using this dataset, the authors train a gradient‑boosted detector that achieves near‑perfect binary classification on oracle labels but shows lower performance on operational labels, and they demonstrate a simple thresholded engine that improves attack‑type attribution accuracy.

By Keifer Lee
arXiv Machine Learning
Oct 2

Towards More Efficient, Robust, Instance-adaptive, and Generalizable Sequential Decision making

The article outlines a Ph.D. research agenda aimed at creating provably efficient and practical algorithms for data‑driven sequential decision‑making under uncertainty. It focuses on reinforcement learning and multi‑armed bandits, targeting applications such as recommendation systems, computer networks, video analytics, and large language models. The work seeks to overcome limitations of existing methods—such as reliance on idealized models, lack of robustness to adversarial perturbations, and poor instance‑dependent performance—by developing algorithms that are more efficient, robust, instance‑adaptive, and generalizable to new environments.

By Zhiyong Wang
arXiv Machine Learning
Oct 2

FairRARI: A Plug and Play Framework for Fairness-Aware PageRank

FairRARI is a new in‑processing convex optimization framework that computes PageRank vectors while satisfying various group‑fairness constraints based on vertex attributes. By formulating PageRank as a variational problem, FairRARI solves a strongly convex problem that guarantees a target fairness level and retains the same asymptotic time complexity as standard PageRank. Experiments on real datasets show that FairRARI outperforms existing methods in utility while achieving the desired fairness across multiple vertex groups.

By Emmanouil Kariotakis, Aritra Konar
arXiv Machine Learning
Oct 2

Vectorized Dynamic Histograms for Sparse Oblique Forests

The paper presents optimizations for Sparse Oblique (SPO) forests in Google’s Yggdrasil Decision Forests, addressing training speed issues caused by runtime sampling of sparse linear feature combinations. By fixing inefficiencies and introducing two new methods—hierarchical AVX2/AVX-512 vectorized histogram filling and runtime‑dynamic histograms—the authors achieve 2–5× speedups for both Gradient Boosted Trees and Random Forests, bringing SPO‑RF training time on par with axis‑aligned RFs. Extensive evaluation on 19 datasets, including up to 10.5 million rows and 1.6 million features, demonstrates these improvements without compromising accuracy.

By Ariel Lubonja, Jungsang Yoon, Haoyin Xu, Yue Wan, Yilin Xu, Richard Stotz, Mathieu Guillame-Bert, Joshua T. Vogelstein, Randal Burns