AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 7

A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment

The paper introduces a human‑in‑the‑loop framework for AI‑assisted scoring of short written responses in a large‑scale national assessment. Using data from two recent test editions with about 5,000 student responses each, the authors validate that AI‑generated scores align moderately to highly with human raters across multiple rubric dimensions. The framework includes a correction workflow that flags cases needing human review, thereby reducing manual workload while maintaining assessment quality.

By Mar\'ia Eugenia Curi, Germ\'an Capdehourat, Isabel Amigo, Magdalena Romano, Rosana Serra, Adri\'an Silveira, Andr\'es Peri
arXiv AI
Sep 7

Robust and Efficient Guardrails with Latent Reasoning

The paper introduces COLAGUARD, a guardrail model that embeds multi-step safety reasoning into a continuous latent space, allowing efficient hidden-state propagation during inference. Compared to existing methods, COLAGUARD achieves an 8.24‑point macro‑F1 improvement over Llama Guard 3 and matches the explicit reasoning baseline GuardReasoner, while delivering a 12.9× speedup and a 22.4× reduction in token usage across ten moderation settings and eight safety benchmarks.

By Siddharth Sai, Xiaofei Wen, Muhao Chen
arXiv AI
Sep 7

Multi-Modal Time Series Prediction via Mixture of Modulated Experts

The paper introduces Expert Modulation, a novel approach for multi‑modal time series prediction that conditions both expert routing and computation on textual signals, thereby providing direct cross‑modal control over expert behavior. Unlike previous methods that rely on token‑level fusion, this mechanism avoids mixing temporal patches with language tokens in a shared embedding space, which can be problematic when high‑quality time‑text pairs are scarce or when time series characteristics vary widely. Experiments and theoretical analysis demonstrate that Expert Modulation yields strong improvements over existing multi‑modal forecasting techniques.

By Lige Zhang, Ali Maatouk, Jialin Chen, Karthik Charan Konduri, Leandros Tassiulas, Rex Ying
arXiv Machine Learning
Sep 7

Interpretability for Turing Machines

The paper demonstrates that the interpretability method known as susceptibilities, originally used for neural networks, can detect algorithmic structure in Turing machines by examining the local loss landscape of a learning problem for noisy Turing machines. It proves that symmetries and path separation in a Turing machine’s algorithm produce permutation symmetries and low‑rank blocks in the susceptibility matrix. Empirical studies on deterministic finite automata show that algorithmic features can be recovered through principal component analysis and clustering in susceptibility space.

By Billy Snikkers, Rumi Salazar, Daniel Murfet, Will Troiani
arXiv Machine Learning
Sep 7

Resilience Beyond Stationary Client Unavailability: Unlocking Efficient and Unbiased Federated Learning

The paper introduces FedSWE, a federated learning algorithm designed to handle non‑stationary and heterogeneous client availability without requiring prior real‑time knowledge of which devices are online. FedSWE compensates for missed computations, stabilizes global updates, and mixes local updates through implicit gossiping, all while adding only modest memory and computational overhead. The authors prove that FedSWE converges to a stationary point for non‑convex objectives and achieves linear speedup in certain scenarios, and they validate these claims with experiments on real‑world datasets featuring diverse client unavailability patterns.

By Ming Xiang, Stratis Ioannidis, Edmund Yeh, Carlee Joe-Wong, Lili Su
arXiv Machine Learning
Sep 7

Single-Query Black-Box Calibration Auditing via Logit Bias

The paper introduces a method for auditing the calibration of large language models (LLMs) that only exposes a logit_bias parameter. By mathematically manipulating this parameter, the authors can evaluate exact probability thresholds with a single query per sample, enabling a provably consistent estimator of True Calibration Error for binary tasks. This approach offers an efficient framework for auditing black‑box foundation models despite limited access to continuous output probabilities.

By Roman Plaud, Antoine Saillenfest, Matthieu Labeau, Thomas Bonald, Willem Waegeman
arXiv Machine Learning
Sep 7

Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation

The paper presents a two-level framework for scalable trade‑up recommendation. Level 1 distills large‑language‑model reasoning into a compact, non‑generative student that classifies product pairs using only precomputed embeddings, achieving high AUC on a benchmark. Level 2 applies product‑type test‑time training to fine‑tune lightweight adapters, further improving performance while keeping inference fast and inexpensive.

By Siliang Liu, Mohammad Ghasemi, Sapan Patel, Amin Banitalebi-Dehkordi
arXiv Machine Learning
Sep 7

Low-Latency Spell Correction for Japanese Music Search Queries

The paper introduces a compact BART-based sequence‑to‑sequence model for low‑latency spell correction of Japanese music search queries, addressing challenges posed by four writing scripts. It employs a script‑aware synthetic misspelling pipeline that blends keyboard‑layout models, phonetic confusion priors, consonant alternations, and kana case errors, and normalizes mixed‑script titles to a single canonical script to reduce hallucinations. Experiments show the model achieves 41.09% exact‑match accuracy and 11.62% CER, outperforming edit‑distance baselines while keeping inference latency under 4 ms on a single GPU.

By Anshul Garg, Pavni Tandon, Karan Bhukar, Tanmay Khandelwal, Ujjal Kumar Dutta
arXiv Machine Learning
Sep 7

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

The paper reports a privacy breach in a two-node split‑LLM training system where the returned gradient reveals which data rows were real, despite the system passing standard privacy checks. By exploiting the fact that decoy rows produce zero gradients, an attacker can identify real rows with 100% accuracy across multiple runs. The authors demonstrate that adding gradient clipping and noise can mitigate the leak, but the system remains vulnerable to several untested attack vectors.

By Georgios Politis, Evangelos Pappas
arXiv Computation and Language
Sep 7

MoirfEolas and Cr\'iochScore: Developing Resources for and the Evaluation of Tokenization Alignment with Irish Morphology

The paper introduces MoirfEolas, a dataset of over 35,000 Irish words annotated with their morphological components, and CríochScore, a metric that measures how well tokenization aligns with these morphological boundaries. Using CríochScore, the authors evaluate common tokenization algorithms and find that the Unigram Language Model best aligns with Irish morphology. They also discuss trade‑offs between morphological alignment, compression, and vocabulary efficiency, offering practical guidance for Irish NLP development.

By Jane Adkins, Abigail Walsh, Brian Davis, Elaine U\'i Dhonnchadha
arXiv Computation and Language
Sep 7

KCSAT-ML: Probing Reasoning Models with Nationwide-Cohort Human Difficulty

KCSAT-ML is a new benchmark built from 664 Korean College Scholastic Ability Test mathematics problems, including a 339‑item core set with official per‑item error rates from nationwide cohorts of hundreds of thousands of examinees. The benchmark introduces the Difficulty‑aligned Reasoning Gain (DRG) metric, which evaluates whether a model’s mistakes align with items humans find hard or easy, revealing distinct patterns in how vision‑language and large language models perform across difficulty levels. The dataset and code are publicly available at https://github.com/naver-ai/KCSAT-ML.

By Sanghee Park, Geewook Kim, Kee-Eung Kim
arXiv Computer Vision
Sep 7

FailSAE: Towards Interpretable Failure Prediction for Vision-Language Models via Sparse Autoencoders

The paper introduces FailSAE, a method that uses Sparse Autoencoders to predict failures in vision‑language models (VLMs) such as CLIP. By treating failure prediction as a classification over sparse SAE latent activations and employing a three‑stage training pipeline, the approach yields higher prediction accuracy than existing confidence‑score or auxiliary‑classifier baselines. Analysis shows that the SAE captures class‑specific concepts and reveals a shift toward ambiguous or style‑related concepts during failures, offering insights for runtime failure recovery.

By Jie Ma, Zongxi Liu, Yi Zhu
arXiv Computer Vision
Sep 7

An Evaluation Framework for Generating Multi-View Images of a Person in a Scene

The paper introduces a framework for generating multi‑view images of a person within a natural scene, addressing the scarcity of paired multi‑view datasets for human subjects. It evaluates existing diffusion‑based image‑editing models and finds they often hallucinate head‑turn angles, leading to inconsistent backgrounds. To overcome this, the authors propose the Head Scene Rotation Difference (HSRD) metric, which separates camera movement from head pose changes and enables reliable assessment of 3D spatial parallax for constructing high‑quality synthetic datasets.

By Mahir Majid, Young Kyung Kim, Guillermo Sapiro
arXiv Computer Vision
Sep 7

SeamFlow: Structure-Aware Flow Matching on Edge Probabilities for Artist-Like UV Unwrapping

SeamFlow is a new generative framework for 3D surface cutting and UV unwrapping that reformulates the discrete mesh‑cutting problem as continuous flow matching in a high‑dimensional edge‑probability space. By learning a deterministic mapping from a Gaussian prior to a target seam‑probability distribution and using an evolution network to couple local topological tokens with global shape priors, SeamFlow guides smooth probability flow through ODE solving. Compared with existing autoregressive generative methods, SeamFlow improves topology awareness, eliminates 3D spatial projection errors and artificial sequential‑order bias, and achieves exceptional semantic coherence with remarkably low parameterization distortion.

By Yuming Zhao, Zangyueyang Xian, Qijian Zhang, Rendong Liang, Qin Jia, Ying He, Junhui Hou
arXiv Computer Vision
Sep 7

Weather-Conditioned Depth Anything

Weather-Conditioned Depth Anything (DA‑W) is a new framework that enhances monocular depth estimation models, like the Depth Anything series, to perform robustly under adverse weather conditions such as fog, rain, snow, and low‑light. It achieves this by disentangling style from content: a Style Filter extracts weather‑specific embeddings from a curated mix of real and synthetic degradation data, which are then injected into the backbone via a lightweight, zero‑initialized adapter. The adapter is trained with pseudo‑label distillation and alignment, enabling a single unified model to adapt to diverse weather scenarios while preserving its generalization on clean data, and it achieves state‑of‑the‑art performance with an average 3.7% improvement in AbsRel on weather benchmarks.

By Zhaoming Xu, Chan-Wei Hu, Kuan-Ru Huang, Zihao Zhu, Renjie Li, Yang Zhou, Zhengzhong Tu
arXiv Computer Vision
Sep 7

PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization

PAPT++ is a risk‑aware adversarial generation‑training framework designed to improve single domain generalization. It learns diverse semantic reference images per class and uses them as denoising targets in classifier‑guided diffusion synthesis, thereby generating challenging yet semantically consistent samples. These samples are iteratively combined with source data to update the classifier, progressively exposing it to difficult variations and enhancing generalization performance on standard benchmarks.

By Zhipeng Xu, De Cheng, Xinyang Jiang, Lingfeng He, Huaijie Wang, Dongsheng Li, Nannan Wang, Xinbo Gao
arXiv Computer Vision
Sep 7

InterSing: Explicit Interaction Dynamics for 3D Duet Singing Animation and Beyond

InterSing is a framework that generates realistic 3D head animations for duet singing by modeling the sparse, rhythm‑dependent interactions between performers. It introduces interaction logits—a weakly supervised, interpretable latent representation of cross‑performer engagement—and uses them to condition an interaction‑aware diffusion model driven by audio and interaction dynamics. The approach enables unified multi‑mode generation, producing coordinated behavior, independent motion, and smooth transitions, and it generalizes to multi‑singer performances with intuitive control over engagement.

By Yihan Zhou, Zikai Huang, Yuyang Yu, Xuemiao Xu, Cheng Xu, Shengfeng He
arXiv Computer Vision
Sep 7

From Interpretability Methods to Interpretable Models

The paper argues that explainable AI for computer vision has focused too much on developing interpretability methods rather than assessing how interpretable the models themselves are. It proposes a shift toward model-centric evaluation, using existing tools to compare what different models represent and compute, and emphasizes the need to measure whether humans can truly understand these models. The authors review the current toolbox, survey limited model comparison work, draw parallels to systems neuroscience, and outline a future agenda for model-focused XAI.

By Julien Colin, Nuria Oliver, Thomas Serre
arXiv Computer Vision
Sep 7

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Motion-Omni is an end‑to‑end framework that jointly generates spoken dialogue and full‑body motion, producing speech, facial expressions, and hand, upper‑body, and lower‑body movements directly from the hidden states of a language model. The system requires joint training of the language model, speech generator, and motion generator to maintain audio‑motion alignment, and it is supervised using a scalable, model‑agnostic pipeline that pseudo‑labels 422,856 speech‑motion pairs. With a Qwen2.5‑7B‑Instruct backbone, Motion‑Omni‑Q7 achieves near‑cascade performance on motion metrics while being 5.4× faster, and it outperforms other non‑teacher cascades on beat correlation, diversity, and word error rate.

By Chengqian Ma, Wei Tao, Haoyu Zhang, Yiwen Guo
arXiv Computer Vision
Sep 7

Explainable Multimodal Deep Learning Integrating Imaging and Clinical Data for Oral Potentially Malignant Disorder Detection

The paper presents M2-OPMDNet, a multimodal deep learning framework that combines white‑light and autofluorescence intraoral images with structured clinical data to detect oral potentially malignant disorders (OPMDs). Using a prospectively collected real‑world dataset, the model achieved an AUC of 0.952, outperforming unimodal approaches and improving detection of visually subtle lesions. Explainability was provided through SHAP analysis, showing that clinical variables significantly contributed to risk estimation alongside imaging features.

By Ruilin You, Yihan Wang, Jiabin Chen, Cherie Wink, Petra Wilder-Smith, Rongguang Liang, Bofan Song