AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Oct 9

VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding

VAMR is a video agent that handles multiple questions about a single long‑form video using a single shared tool‑use trajectory, rather than treating each question separately. It employs question‑conditioned visual perception, layered multi‑question memory, and a question‑horizon policy optimization to jointly acquire evidence and progressively build a shared understanding. In benchmark tests (LVBench, Video‑Holmes, LongVideoBench), VAMR outperforms prior methods, achieving higher accuracy and fewer reasoning rounds and processed frames.

By Runquan Gui, Hanzhu Chen, Zehao Wang, Hanxin Zhu, Xin Li, Zhibo Chen
arXiv AI
Oct 9

Neuro-Memory Fuzzy Inference System for Mimicking Human-like Car Following Behavior

arXiv:2610. 11252v1 Announce Type: cross Abstract: This study presents the Neuro-Memory Fuzzy Inference System (NeMeFIS), a hierarchical machine learning architecture that asymmetrically models acceleration and deceleration in car following behavior by integrating five human memory types procedural, working, episodic, semantic, and declarative.

By Nazmul Haque, Md Asif Raihan. Md. Hadiuzzaman
arXiv AI
Oct 9

MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval

MSGAT introduces a Multi-head Spiking Graph Attention Network that models structural relationships for image-text retrieval using spike-driven graph reasoning. It employs dynamic attention heads to capture complementary relational patterns and a two-branch multi-granularity fusion framework. The proposed Sim-Fuse strategy aligns coarse- and fine-grained matching relations in similarity space, avoiding interference between sparse spike representations and continuous global features, achieving state‑of‑the‑art performance on Flickr30K and MSCOCO with significant energy savings.

By Xintao Zong, Wenxuan Liu, Jianhao Ding, Zhaofei Yu, Tiejun Huang
arXiv AI
Oct 9

Same Outcome, Different Evidence: Intent Recovery in LLM Safety Evaluation

The paper proposes adding an operative understanding rate (UR) alongside the traditional attack success rate (ASR) to better assess large language model safety. UR measures whether a model’s response both identifies and treats the evaluated task as the task to be answered, revealing variations in intent recovery that ASR alone masks. Experiments show that explicit prompts improve recovery rates while ASR remains unchanged, and that high recovery can still coincide with harmful assistance, underscoring the need for joint reporting.

By Haitong Jiang, Chunlin Liu, Sihan Tang, Chan Wu, Xiaoqing Su, Yuhong Feng
arXiv AI
Oct 9

A Geometric Approach to Soft Actor-Critic with Zonotopes for Locomotion Learning

The paper introduces GeZo‑SAC, a variant of Soft Actor‑Critic that incorporates geometric representations (zonotopes) to adapt critic pessimism based on state and action. Each critic predicts a zonotope whose width is subtracted from the critic value as a pessimistic offset, and the disagreement between critics is measured and used to blend the critics via a log‑sum‑exp aggregation. Experiments on MuJoCo‑v5 locomotion tasks show GeZo‑SAC achieving the highest mean return on Ant‑v5 and Hopper‑v5, while also reducing actuator work and action effort and maintaining near‑zero overestimation frequency.

By Panagiotis Roditis, Panagiotis P. Filntisis, Petros Maragos
arXiv AI
Oct 9

ReSI: Recursive Safety Improvement toward Resistant and Resilient AI

ReSI is a recursive safety improvement framework that iteratively applies diverse red‑teaming methods to identify vulnerabilities in AI models, develops training recipes, and selects the update with the greatest safety gain while retaining capability. Across four dense and mixture‑of‑experts models, ReSI matches or surpasses frontier models on both in‑distribution and out‑of‑distribution safety benchmarks, and outperforms alignment baselines on nearly all safety evaluations. The framework notably reduces the mean X‑Teaming attack success rate from 86.01% to 31.45%, outperforming GPT‑5.6‑Luna’s 56.69% and demonstrating stronger resilience to unseen attacks.

By Jingnan Zheng, Dongcheng Zhang, Yi Zhang, Ming Zhang, Qiaosheng Zhang, Youbang Sun, An Zhang, Xiangnan He, Tat-Seng Chua, Xia Hu, Bowen Zhou, Chaochao Lu, Xiang Wang
arXiv AI
Oct 9

Prediction-Powered Data Fusion for Treatment Effect Estimation

The paper introduces a framework that fuses data from a small randomized controlled trial (RCT) with a large observational study (OBS) to estimate treatment effects without imposing special assumptions on the OBS. It presents an ATE estimator called AIPW‑Fusion, which uses closed‑form weights and confidence intervals, and two conditional ATE (CATE) learners, DR‑Fusion and R‑Fusion, that preserve unbiasedness while borrowing precision from the OBS. Experiments validate the proposed methods.

By Yonghan Jung, Shu Yang
arXiv AI
Oct 9

Predicting Alignment Generalization with Value Representations

The paper introduces the task of predicting alignment generalization, which measures how fine‑tuning a language model to follow a specific value affects its behavior across a broad set of other values. Through a large‑scale study of 66 values, the authors show that representations derived from model activations when applying values in context outperform textual description‑based methods, achieving a correlation of 0.45 versus 0.05. They further demonstrate that these representations can predict downstream robustness and help construct a taxonomy of LLM values based on empirical generalization dynamics.

By Andy Liu, Mehar Bhatia, Karolina Stanczak, Mona Diab, Vered Shwartz, Daniel Fried
arXiv AI
Oct 9

Towards Reasonable Concept Bottleneck Models

The paper introduces CREAMs, a flexible framework for Concept Bottleneck Models that allows explicit encoding of concept‑concept and concept‑task relationships such as mutual exclusivity, hierarchy, and correlations. CREAMs can optionally use a regularized side‑channel to compensate for incomplete concept sets, maintaining competitive performance while encouraging concept‑grounded predictions. The authors also propose a new metric to assess interpretability when predictions partially rely on the side‑channel, and demonstrate that CREAMs support efficient interventions, avoid concept leakage, and match black‑box performance even with missing concepts.

By Nektarios Kalampalikis, Kavya Gupta, Georgi Vitanov, Isabel Valera
arXiv AI
Oct 9

APEX: Assumption-free Projection-based Embedding eXamination Metric for Image Quality Assessment

APEX is a new image quality assessment framework that eliminates assumptions by using the Sliced Wasserstein Distance as a similarity measure. It is embedding‑agnostic, scalable to high‑dimensional spaces, and employs open‑vocabulary models CLIP and DINOv2 for feature extraction. Benchmarks show that APEX is more robust to visual degradations and remains stable across different datasets, including out‑of‑domain ones.

By Caterina Gallegati, Monica Bianchini, Franco Scarselli, Vittorio Murino, Barbara Toniella Corradini
arXiv Machine Learning
Oct 9

When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry

The paper demonstrates that routing information logged by Mixture-of-Experts (MoE) language models can be used to infer whether a specific example was part of a model’s fine‑tuning data. By combining router telemetry with standard output signals and training a membership classifier on shadow models, the authors achieve higher true‑positive rates across multiple MoE architectures and data domains. The study shows that this privacy leakage persists under various fine‑tuning regimes and even with limited telemetry, indicating that router logs can become an additional privacy risk for deployed MoE models.

By Yixin Tan, Jiayang Liu, Lu Sun, Yuke Hu, Zheng Li, Rui Wen
arXiv Machine Learning
Oct 9

Teaching PPG How not Who: Fixed-Effects Distillation from ECG

The paper investigates how ECG data used to train photoplethysmography (PPG)-only models primarily teaches the model to recognize individual identity rather than cardiovascular state changes. By removing each recording’s mean (fixed‑effects distillation), the authors eliminate the identity signal, leading to a substantial increase in state agreement and improved within‑person performance across multiple models and datasets. The study demonstrates that conditioning on the recording itself directs distillation toward the dynamic changes that wearables are designed to capture.

By Zhongli Wu, Zhuangzhi Gao, Yuankai Wang, Gregory Y. H. Lip, Bilal H. Kirmani, Yalin Zheng
arXiv Machine Learning
Oct 9

Low-rank tensor structure of precipitation and its application to satellite-reference merging

The study shows that daily precipitation over the contiguous United States can be represented as a low‑rank spatiotemporal tensor, and that preserving native spatial and temporal modes via CANDECOMP/PARAFAC factorization yields more accurate reconstructions than treating daily fields independently. Building on this, the authors developed TMerge, a tensor‑based framework that merges satellite precipitation with sparse reference observations through shared low‑rank factors. Applied to the IMERG Final Run product, TMerge raised correlation from 0.53 to 0.85 and cut root‑mean‑square error and mean absolute error by 48.2 % and 29.3 %, outperforming linear bias correction, quantile mapping, and neural networks across seasons, intensity regimes, and regions, especially in coastal areas.

By Ryan Solgi, Rohan Shankar, Hugo A. Loaiciga
arXiv Machine Learning
Oct 9

Ranking Prior Alignment for Credit Risk Modeling: When Do External Priors Matter?

The paper introduces Ranking Prior Alignment, a model‑agnostic framework that injects external ranking priors—derived from domain experts, teacher models, or large language models—into credit‑risk scoring models via a temperature‑scaled KL divergence loss. It unifies neural (MIL attention) and tree‑based (XGBoost) architectures under a single loss formulation, requiring no external model at inference and tolerating significant annotation noise. Experiments on a large industrial dataset and public Amex data show consistent AUC gains across multiple model families, with the benefits increasing when data, model capacity, or feature quality are limited.

By Qiye Lu, Jiang Ji, Liang Zhang
arXiv Machine Learning
Oct 9

SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting

SACQ introduces a structured prediction head for long‑horizon time‑series forecasting that replaces the traditional flatten readout. It uses a two‑stage decoding process: a coarse patch‑grid scaffold is first generated, then each future position is refined via cross‑attention over historical memory, with a learned gate merging the corrections. A batch‑adaptive scaled log‑cosh loss further stabilizes training under long horizons and noisy labels, leading to superior MSE/MAE performance across multiple backbones with minimal overhead.

By Guo Cheng, Zhengzhuo Xu, Chenchen Jing, Jingyi Hou
arXiv Machine Learning
Oct 9

CARE: A Lightweight Plug-in Gated Correction and Uncertainty-aware Module for Long-term Time Series Forecasting

CARE is a lightweight plug‑in module that can be added to any deterministic time‑series forecaster. It resamples historical data to align with the forecast horizon, learns residual corrections, and applies scale‑aware bounded updates controlled by per‑coordinate sigmoid risk gates. The module is trained with a multi‑objective loss to improve accuracy, track residuals, align risk, and anchor the base model, and it demonstrates accuracy gains on eight benchmarks with minimal extra parameters or latency.

By Guo Cheng, Changlong Lv, Jingyi Hou
arXiv Machine Learning
Oct 9

Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models

The paper presents a preference-driven unlearning framework for few-step distilled text-to-image diffusion models. It identifies that standard Direct Preference Optimization (DPO) fails on these models due to altered generation dynamics and proposes a modified formulation aligned with few-step properties. The method enables direct concept removal while preserving desirable capabilities, and is evaluated on identity, NSFW, and object-level unlearning tasks with strong forgetting and retention results.

By Gaurav Patel, Jun Fang, Greg Ver Steeg, Qiang Qiu, Sravan Sripada
arXiv Machine Learning
Oct 9

GHARP: Real-time Gaussian Head Animation from Large-scale Reconstruction Prior

GHARP is a two‑stage method that animates 3D human heads in real time using a large‑scale reconstruction prior. The first offline identity stage builds a compact Gaussian representation of a subject’s geometry and appearance, while the lightweight runtime animation stage predicts expression‑dependent residuals in a semantically structured latent space. A body alignment network further refines the animation by aligning body pose and clothing, enabling state‑of‑the‑art quality on the Ava‑256 benchmark with significantly faster performance and fewer Gaussians.

By Ali Benlalah, Sepehr Johari, Patricia Vitoria, Armin Kappeler, Artem Sevastopolsky, Alexander Jung, Gabriele Fanelli, Kevin Mader, Manuel Breitenstein, Claudia Pl\"uss, Jan R\"uegg, Simon Biland, Thomas Etterlin, Dmitry Kostiaev, Mathias Deschler, Brian Amberg, Sebastian Martin
arXiv Machine Learning
Oct 9

Learning What to Trust in Multimodal Learning under Noisy Supervision

The paper introduces REFINE, a multimodal label‑noise detection framework that leverages both fused and unimodal representations to identify trustworthy examples in noisy supervision settings. By constructing discriminative eigenvectors for target and background classes, REFINE selects trusted representation spaces and measures alignment of instance representations within these spaces, combining the resulting subsets to provide cleaner supervision for the multimodal classifier. Experiments across various tasks show that REFINE outperforms baseline methods, improving model generalization while reducing the impact of mislabeled data.

By Jiashuo Zou, Xiaobo Xia
arXiv Machine Learning
Oct 9

BRACE: Differential Privacy for Dense Associative Memory with LSR Energy

The paper introduces BRACE, a differentially private retrieval algorithm for log-sum-ReLU dense associative memory (LSR-DAM). BRACE adaptively corrects boundary-sensitive perturbations to control their cumulative effect during retrieval, achieving minimax optimal error rates that are independent of dimension and depend optimally on inverse temperature and retrieval horizon. The authors also prove central limit theorems for uncertainty quantification and demonstrate the method’s accuracy against baseline privacy approaches in numerical experiments.

By Chang Qu, Zhaoyang Shi