AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Sep 24

Helping Customers in Distress: An LLM-powered Agent that Converses, Probes, and Routes

The paper presents an AI‑powered triaging agent for banks that uses large language models to conduct multi‑turn conversations, ask relevant questions, and classify customer cases for accurate routing to specialist teams. The system is integrated with policy, safety guardrails, and reasoning frameworks, and its performance is evaluated using synthetic digital twins that simulate realistic, labeled dialogues based on historical data. Results show a 30.6% increase in classification accuracy and high satisfaction from subject‑matter experts, demonstrating the effectiveness of targeted probing for scalable banking operations.

By Alankar Atreya, Stefan Sylvius Wagner, Devesh Batra, Robert Hankache, Cristovao Iglesias Jr, Patrick Sinclair, Giulio Pelosio, Michael McMillan, Greig A. Cowan, Raad Khraishi
arXiv Machine Learning
Sep 24

PR-Smoother: Simulator-Preserving Non-Gaussian Smoothing for Data Assimilation

PR‑Smoother is an amortized smoothing method that preserves the explicit use of a prescribed simulator in both the evidence lower bound and the variational family. It learns only future‑conditioned corrections to the simulator’s rollout, yielding a non‑Gaussian smoothing distribution that can jointly infer state, parameters, and sensor bias from observations alone. The approach recovers the exact smoother in deterministic and linear‑Gaussian limits and has been shown to capture multimodal posteriors in Lorenz‑96 and scale to 16,384‑dimensional Kolmogorov flow.

By Yuta Tarumi
arXiv Machine Learning
Sep 24

CORE-STACK+: Meta-Learning for Deep Stacked Generalization

CORE-STACK+ is a new meta‑learning framework for deep stacked generalization that tackles two key problems in heterogeneous vision ensembles: prediction‑space multicollinearity and calibration collapse. It introduces a four‑step preconditioning pipeline—kernelized redundancy filtering, a lightweight differentiable meta‑feature gate, a spectrum‑adaptive ridge penalty, and a Laplace‑approximate Bayesian blender—to jointly improve conditioning and calibration. Across six vision benchmarks, CORE‑STACK+ boosts accuracy, reduces model count and inference cost, and significantly lowers expected calibration error compared to existing methods.

By Noor Islam S. Mohammad
arXiv Machine Learning
Sep 24

When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations

The paper introduces FAPE, a four‑stage framework for evaluating the post‑processing fairness intervention ThresholdOptimizer across eight diverse domains, including criminal justice, finance, healthcare, and education. It reports that the intervention reduces disparity in most high‑disparity cases but can worsen fairness when baseline disparities are low, and that a single deployment‑time audit is unreliable without continuous monitoring and baseline‑disparity screening.

By Nithin Raghava Ramachandra Narla
arXiv Machine Learning
Sep 24

Learning Risk Scores Robust to Unobserved Confounders

The paper introduces a method for learning risk scores that remain reliable even when historical data contain unobserved confounders. By treating propensity weights as uncertain and applying sensitivity analysis with Wasserstein distributionally robust optimization, the authors formulate a robust learning problem solvable via an exponential cone program. Experiments on semi‑synthetic UCI data show the approach improves calibration by up to 29.2% over traditional benchmarks and 11.1% over the state of the art, without harming other performance metrics.

By Ryan Edmonds, Yingxiao Ye, Sina Aghaei, Andr\'es G\'omez, \c{C}a\u{g}{\i}l Ko\c{c}yi\u{g}it, Phebe Vayanos
arXiv Machine Learning
Sep 24

What Converges in the Platonic Representation Hypothesis? Structure over Geometry

The paper investigates the Platonic Representation Hypothesis, which posits that more capable models converge toward shared representations. By distinguishing relational structure (which samples are related) from metric geometry (quantitative relations like distances), the authors develop a controlled $2 imes2$ framework to evaluate both aspects at local and global scales. Their findings show that relational structure consistently converges across vision‑language and video‑text models, while metric geometry converges much more weakly, a pattern that persists even when using a Riemannian metric approximation.

By Junwon You, Mihyun Jang, Sangwoo Mo, Jae-Hun Jung
arXiv Machine Learning
Sep 24

Learning Where to Look: A Shared Relative-Alignment Module for Time-Series Forecasting and PPG-to-Vital-Sign Reconstruction

The paper introduces ROOSTER, a shared relative‑alignment module that learns how to map a condition sequence to a target sequence for both PPG‑to‑vital‑sign reconstruction and long‑horizon multivariate time‑series forecasting. ROOSTER uses a learnable periodic‑comb bias over the target‑condition offset, allowing it to discover identity alignment or seasonal lags and report the chosen correspondence. Experiments show that ROOSTER outperforms existing baselines on heart‑rate and respiratory‑rate reconstruction from wrist‑worn photoplethysmograms, and achieves the lowest horizon‑averaged MSE on four forecasting benchmarks, outperforming the underlying forecasting model in most dataset‑horizon settings.

By Ragamayi Puli, Shunya Nagashima
arXiv Machine Learning
Sep 24

VCMM: Variance-Calibrated Momentum for Multimodal Learning

VCMM: Variance-Calibrated Momentum for Multimodal Learning proposes a new optimizer that adapts momentum based on modality-specific gradient dynamics. It estimates minibatch noise and temporal drift online, using a Kalman-inspired controller to set modality-specific momentum and applies bias correction for the first moment. Experiments on four multimodal benchmarks show consistent improvements with modest training overhead.

By Zhongjing Gu, Chenyang Huang, Yufa Feng, Chong He, Qinxu Ding, Yiming Cui
arXiv Machine Learning
Sep 24

Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections

The paper investigates minimal‑norm interpolation and λ2‑regularized logistic‑loss minimization for binary classification using univariate two‑layer ReLU networks. It provides exact geometric characterizations of optimal classifiers, showing that unpenalized hidden‑layer biases yield continuous piecewise‑affine functions that tightly follow label switches, while penalized biases produce a unique, sparsest classifier with a single kink per same‑label segment. Adding a free affine skip connection does not change these function‑space solutions but guarantees that every KKT point becomes globally optimal, eliminating suboptimal KKT points that can arise without the skip connection.

By Karolina Drabik, Ben Lewis, Antoni Puch, Etienne Boursier, Piotr Hofman, Matthias Englert, Ranko Lazi\'c
arXiv Machine Learning
Sep 24

The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity

The study investigates whether sensory-aligned receptive fields provide computational benefits beyond mere resource efficiency in recurrent networks of Expressive Leaky Memory neurons. Across auditory and event-based visual classification tasks, receptive fields aligned with task-relevant sensory coordinates improve test accuracy compared to budget-matched random fields, but this advantage disappears when coordinates are scrambled or irrelevant. The benefit diminishes as neuronal expressivity increases, and generic synaptic sparsity regularization only partially recovers performance, indicating that structured receptive fields act as a computational prior beyond sparsity alone.

By Agnese Adorante, Aaron Spieler, Anna Levina
arXiv Machine Learning
Sep 24

Quantifying the Occult: A Comparative Study of Hindu and Buddhist Deities Using Machine Learning Methods

The paper presents a dual‑matrix computational framework that quantifies morphological and theological differences among 196 Hindu and Vajrayana Buddhist esoteric deities. It uses a Gower distance matrix with a new Cardinality Weighting algorithm for physical form and dense vector embeddings from LLMs for theological function, revealing how visual forms can mask shared functions and how high‑cardinality symbols cluster orthodox and Tantric entities. The study demonstrates near‑identical coordinates for the Hindu Chinnamasta and Buddhist Chinnamunda, and releases the architecture as an open‑source tool for Digital Humanities research.

By Ankit Bhattacharjee
arXiv AI
Sep 24

Safety Nudges: User-Facing Interventions for Real-Time AI Risk Awareness

The paper introduces Safety Nudges, a browser-based tool that displays lightweight, in situ flags when a conversational AI exhibits risky behavior such as hallucination or overconfidence. In a two‑week field study with 45 frequent chatbot users, participants reported that the nudges were useful, clear, and minimally disruptive, and most felt more aware of potential AI harms. However, increased awareness did not automatically translate into measurable changes in user behavior, underscoring the need for relevance, calibration, and user control in nudge design.

By Varshini Elangovan, James Wedgwood, Chhavi Yadav, William Agnew, Sauvik Das, Virginia Smith