AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computation and Language
Oct 19

Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes

arXiv:2606.27210v2 Announce Type: replace Abstract: We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce...

By Jeremias Ferrao, Niclas M\"uller-Hof, Iustin S\^irbu, Traian Rebedea, Yftah Ziser
arXiv AI
Oct 19

Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize

The paper introduces Neural Expansion, a geometric framework that explains how deep neural networks generalize by expanding directionally-selective ray coverage in input space. It defines Generalization Intervals (GIs) to measure stable prediction regions around training samples and shows through experiments that training progressively enlarges this coverage, especially along directions with low Jacobian singular values. This expansion accounts for correct test predictions, adversarial robustness, and handling of mislabeled or out-of-distribution data, offering a mechanistic complement to existing global theories.

By Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino, Canlin Zhang, Xiuwen Liu
arXiv AI
Oct 19

Reason-guided Human Reward Modeling for Preference-based Reinforcement Learning

The paper introduces a reason-guided preference-based reinforcement learning framework that learns a human reward model from trajectory-level feedback. It augments traditional preference labels with reason tags that explain why a behavior is preferred, thereby capturing both human choices and their underlying rationale. By combining the learned human reward with the environment reward during policy optimization, the agent can improve task performance while producing more interpretable, human-aligned, and preference-sensitive behavior.

By Anshul Nayak, Shahil Shaik, Jonathan Smereka, Yue Wang
arXiv AI
Oct 19

Moral Sycophancy in Vision Language Models

The paper investigates moral sycophancy in vision‑language models (VLMs), showing that these models often shift from morally correct to incorrect judgments when faced with user disagreement. Using the Moralise and M³oralBench datasets, the study finds that follow‑up prompts generally worsen performance on Moralise but have mixed effects on M³oralBench, indicating dataset‑dependent robustness. Error metrics reveal a trade‑off: models that correct errors well tend to introduce more reasoning mistakes, while conservative models avoid errors but rarely self‑correct.

By Shadman Rabby, Md. Hefzul Hossain Papon, Sabbir Ahmed, Nokimul Hasan Arif, A. B. M. Ashikur Rahman, Irfan Ahmad
arXiv AI
Oct 19

Missing value imputation with adversarial random forests - MissARF

The paper introduces MissARF, a new imputation technique that uses adversarial random forests (ARF) for density estimation and data synthesis. MissARF can perform both single and multiple imputations by sampling from the conditional distribution estimated by ARF, conditioned on observed values. Experiments show that MissARF achieves imputation quality comparable to leading methods while offering faster runtimes and no extra cost for multiple imputations.

By Pegah Golchian, Jan Kapar, David S. Watson, Marvin N. Wright
arXiv AI
Oct 19

A Theoretical Analysis of Test-Driven Code Generation

The paper presents a probabilistic framework for two main test‑driven code generation paradigms: selecting code after generation using an execution environment, and generating code conditioned on environment feedback. It formalizes selection heuristics as environment‑aware estimators of code correctness, proving that fuzzy functional similarity estimators provide a stronger inductive bias and higher signal‑to‑noise ratio than functional equivalence estimators. Additionally, it models backprompting as an in‑context approximation of Thompson sampling, deriving a regret bound that explains its limitations due to ambiguous task descriptions, and validates these theoretical insights with experiments on seven open‑weight models across multiple benchmarks, including a new quantum programming benchmark.

By Nicolas Menet, Michael Hersche, Andreas Krause, Abbas Rahimi
arXiv AI
Oct 19

Algorithmic Primitives of Theory of Mind Benchmarks

The paper questions whether Theory of Mind (ToM) benchmarks truly assess theory of mind or merely test parsing and algorithmic state tracking. It introduces PDDL-Mind, a neuro‑symbolic method that parses narratives into the Planning Domain Definition Language (PDDL) and executes candidate actions using predefined update algorithms, yielding program‑verified state traces. Experiments on MMToM, MuMA, and FanToM show that delegating parsing to programs and state tracking to PDDL significantly improves performance, suggesting that current benchmarks mainly penalize unreliable implicit state tracking rather than genuine belief or counterfactual reasoning.

By Qiutong Tony Yi, Wang Bill Zhu, Robin Jia, Jesse Thomason
arXiv AI
Oct 19

MOSAIC: Module Discovery via Sparse Additive Identifiable Causal Learning for Scientific Time Series

MOSAIC is a sparse temporal variational autoencoder that merges causal representation learning with support recovery to identify interpretable latent modules in scientific time series. It leverages regime‑conditioned temporal variation to discover latent variables and uses an additive decoder to recover a sparse set of observations linked to each latent, providing module‑level interpretability. The method offers theoretical guarantees for ANOVA main‑effect identifiability under smooth mixing functions and demonstrates empirical success across diverse domains such as RNA dynamics, solar wind, ENSO climate, industrial process control, and synthetic tokamak data.

By Shicheng Fan, Nour Elhendawy, Jianle Sun, Ke Fang, Kun Zhang, Yihang Wang, Lu Cheng