AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Machine Learning
Aug 28

The Rashomon Effect for Visualizing High-Dimensional Data

The paper introduces the Rashomon set for dimension reduction, a collection of equally good embeddings that preserve high‑dimensional structure. It proposes PCA‑informed alignment to make axes interpretable, concept‑alignment regularization to incorporate external knowledge, and a method to extract trustworthy nearest‑neighbor relationships across the Rashomon set for refined embeddings. These techniques aim to produce interpretable, robust, and goal‑aligned visualizations by leveraging multiple valid embeddings instead of a single one.

By Yiyang Sun, Haiyang Huang, Gaurav Rajesh Parikh, Cynthia Rudin
arXiv Computation and Language
Aug 28

Towards Interpretable Depression Detection: Linking Acoustic Features to DSM-5 Indicators

The paper proposes a transparent framework that links speech acoustic features—such as pitch variability, pauses, and speech tempo—to DSM‑5 indicators of depression, offering interpretable, indicator‑level outputs instead of opaque black‑box models. It runs locally on commodity hardware to preserve privacy and has been preliminarily evaluated on the DAIC‑WOZ dataset, showing consistent associations between acoustic cues and DSM‑5 indicators of psychomotor change and concentration difficulty. Future work aims to validate the approach on longitudinal data and expand multimodal integration while keeping edge constraints.

By Jonas L\"anzlinger, Katharina O. E. M\"uller, Burkhard Stiller, Bruno Rodrigues
arXiv Computation and Language
Aug 28

STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation

The paper introduces STAR, a metric that measures sentence-level alignment between source and target documents in document-to-document machine translation. Using STAR, the authors develop StarPO, a preference‑optimization framework that ranks translation hypotheses by structural quality and applies a dynamic alignment mask to focus learning on misaligned segments. Experiments on news and literary data show that StarPO improves both translation quality and structural integrity, enabling small models to outperform large proprietary systems such as GPT‑4o while remaining more token‑efficient.

By Yichen Dong, Hao Wang, Junhui Li, Linlong Xu, Longyue Wang, Weihua Luo
arXiv Computation and Language
Aug 28

Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG

The paper investigates whether incorporating an evidence-support signal into retrieval evaluation for retrieval‑augmented generation (RAG) improves downstream decision‑making. Across multiple benchmarks and a TREC RAG 2025 setting, the evidence signal alters retriever rankings but its benefits vary: it does not consistently enhance retriever training, its usefulness for system selection depends on generator instructions, and it does not reliably predict answer quality on unseen topics. Human filtering of evidence‑rich passages preserves useful content, yet evaluators disagree on whether this improves final answers, indicating that evidence‑aware evaluation alone does not guarantee better downstream outcomes.

By Utshab Kumar Ghosh, Debayan Mukhopadhyay, Shubham Chatterjee
arXiv Computation and Language
Aug 28

CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation

CultureVidBench is a new benchmark that evaluates how well text‑to‑video generation models capture cultural details. It contains 1,000 prompts spanning 12 countries, 6 continents, 8 cultural regions, and 14 cultural aspects, grouped into material culture, social practice & performance, and ritual & ceremony. Human studies and automated assessments show that while current models perform well on semantic adherence and visual quality, they often miss fine‑grained cultural details, especially for underrepresented regions and multimodal cues.

By Xianjing Han, Yuhan Su, Yang Deng, Dong Ma, Wee Peng Tay, Bin Zhu
arXiv Computer Vision
Aug 28

GeoMAD: Geometry-Aware Multi-View Anomaly Detection via Deformable Fusion and Distributional Alignment

GeoMAD is a multi‑view anomaly detection framework that fuses multiple camera viewpoints while maintaining geometric awareness and scalability to multi‑class industrial settings. It introduces a Cross‑view Deformable Fusion Module (CDFM) that learns view‑pair‑specific sampling offsets on 2D feature maps, enabling hierarchical cross‑view correspondence without camera calibration or voxel construction. Additionally, Distributional View Alignment (DVA) provides a self‑supervised loss that aligns bottleneck distributions across views, ensuring global consistency without pixel‑level correspondence. Together, CDFM and DVA achieve geometry‑aware, distribution‑consistent fusion and demonstrate strong detection and localization performance on Real‑IAD and MANTA‑Tiny datasets.

By Shang-Fu Chen, Jhih-Ciang Wu, Kuan-Chuan Peng, Wen-Huang Cheng, Kai-Lung Hua
arXiv Computer Vision
Aug 28

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher

Self-OPD introduces a teacher‑free on‑policy distillation framework for flow matching models, using the student’s own exploration to generate step‑wise supervision. At each timestep the deterministic next‑state prediction is branched into multiple stochastic SDE candidates, rolled out, and compared against a deterministic baseline to compute normalized advantages. The velocity field is then optimized with a pull‑push objective that attracts high‑advantage branches and repels low‑advantage ones, while multi‑objective alignment is achieved by fusing normalized scores at the reward level.

By Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He, Siyu Zou, Jinlong Liu, Yunlong Yu, Jian Song, Hao Jiang, Pipei Huang, Bo Zheng
arXiv Computer Vision
Aug 28

Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning

Aphanta is an automated framework that diagnoses how well image editors can produce task‑aligned visual intermediates for multimodal large language models (MLLMs). It evaluates three reasoning conditions—direct, editor‑generated, and idealized intermediate—to distinguish visual potential from practical editor performance across 20 tasks and various editor–MLLM pairs. The study finds that image editing benefits certain tasks like visual cue injection and grounding, but is less reliable for symbol‑sensitive or structural tasks, and demonstrates measurable performance gains with a Qwen pipeline.

By Hengyuan Xu, Wei Cheng, Yumeng Ji, Xuanyang Zhang, Xianfang Zeng, Gang Yu, Xingjun Ma
arXiv Computer Vision
Aug 28

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.

By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv Computer Vision
Aug 28

High-Frequency First: A Two-Stage Approach for Improving Image INR

The paper proposes a two-stage training strategy for Implicit Neural Representations (INRs) that addresses spectral bias by using a neighbor-aware soft mask to emphasize high-frequency details early in training. In the first stage, the mask assigns higher weights to pixels with strong local variations, encouraging the network to focus on fine edges and textures. The second stage transitions to full-image training, and experiments show consistent improvements in reconstruction quality across existing INR methods.

By Sumit Kumar Dam, Mrityunjoy Gain, Eui-Nam Huh, Choong Seon Hong
arXiv Statistics ML
Aug 28

On the approximation of posterior laws in compound loss models by conditional Wasserstein GANs

The paper introduces a conditional Wasserstein GAN to approximate posterior distributions in compound loss models, conditioning on sufficient statistics, prior mean, coefficient of variation, and mixture weights. A single generator can learn the posteriors for both Poisson intensity and Pareto shape parameters across Gamma, inverse‑Gaussian, and lognormal priors. The authors validate the approach with simulation‑based calibration, analytical comparisons, and extensive MCMC, and apply it to extreme natural catastrophe loss data to generate rolling one‑year posterior predictive distributions and assess tail risk under heavy‑tailed severity and prior uncertainty.

By Aleksandar Arandjelovic, Pavel V. Shevchenko, George Tzougas
arXiv AI
Aug 28

Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs

The paper introduces REMI, a framework that treats counterfactual fairness as a relational invariant discovery problem. By learning over paired examples, REMI identifies input regions where fairness is violated and generates interpretable rule-based models—fairness invariants—that can block or relabel unfair predictions without retraining the underlying model. Experiments on symbolic and neural network programs show REMI localizes fairness bugs in over 83% of cases and reduces discriminatory decisions in black-box models by up to 70%.

By Ranit Debnath Akash, Ashish Kumar, Gang Tan, Saeid Tizpaz-Niari
arXiv AI
Aug 28

On Scope Classification and Current Knowledge-Editing Benchmarks: A Negative Result, with INLAY as a Gradient-Free Case Study

The paper reports that existing knowledge‑editing benchmarks cannot evaluate the scope decision—whether a stored edit applies to a query—because they are counterfactual and lack negative examples. Using the gradient‑free editor INLAY, the authors exhaustively test every router action on 1,689 queries across three datasets and find that an oracle router achieves no gain over a static policy, and abstention never wins. The authors attribute this to the structural design of the benchmarks and demonstrate that adding a missing negative condition restores some headroom and allows abstention to win.

By Aditya Pratap Singh
arXiv Machine Learning
Aug 28

SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting

SAGE is a CLIP-based framework that augments vision‑language time series forecasting by incorporating variable‑specific semantic and statistical information. It processes frequency‑enhanced patches and variable tokens through a CLIP text encoder, while a frozen CLIP vision encoder aligns rendered series with temporal representations via a contrastive objective. The approach achieves state‑of‑the‑art accuracy on eight long‑term benchmarks and M4, with ablations showing complementary gains from multimodal alignment and variable‑level knowledge.

By Haizhao Fan, Xinyi Le
arXiv AI
Aug 28

Approved Too Late: Verdict Staleness in LLM-Guarded Self-Adaptive Systems

The paper investigates how approvals issued by a large language model (LLM) guardrail for self‑adaptive systems can become stale between the time of check and the time of use, creating a TOCTOU hazard. It introduces three metrics for verdict freshness, evaluates them across five SAS environments, and proposes the Freshness‑Bounded Shield (FBS) to estimate an approval’s validity horizon without a plant‑dynamics model, reducing expiry rates significantly. The study also audits LLM judges and formulates a freshness contract requiring approvals to remain valid at use time.

By Ilai Shraga, Roei Eshel, Lior Gorelik
arXiv AI
Aug 28

Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing

The study examines how professional English editing influences AI text detectors’ false-positive rates for non-native academic writing. Using 135,389 pairs of original and edited manuscripts, researchers found that detector responses varied widely—some editors increased AI scores while others decreased them—and that score changes correlated with the extent of editing. These results highlight professional editing style as a key confounding factor in AI detection, complicating the distinction between AI authorship and linguistic style.

By Hyeonchu Park, Gahye Jeong, Bugeun Kim
arXiv AI
Aug 28

Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds

The paper introduces GRAPHSU, a graph‑guided selective unlearning method for language models that expands deletion beyond explicitly identified forget seeds. By constructing a weighted support‑route graph and propagating deletion pressure, GRAPHSU applies graded forgetting to high‑risk neighboring examples. Experiments on the TOFU and PISTOL benchmarks with GPT‑2 Medium and Llama‑3.2‑3B‑Instruct show that GRAPHSU achieves the lowest utility‑feasible soft leakage, reducing leakage by up to 49.5 percentage points compared to a seed‑only baseline.

By Waqas Khan, Tabinda Sarwar, Jingyue Cong, Xun Yi, Estrid He
arXiv AI
Aug 28

Not All Eval-Awareness Is Equal: Capabilities Framing Predicts Compliance

The paper investigates how different framings of a model’s evaluation awareness—whether it is seen as a capabilities cue, a safety cue, both, or neither—affect its compliance with instructions. Experiments on Qwen3-32B using the FORTRESS dataset show that a capabilities‑framed awareness leads to significantly higher compliance (a 24–46 percentage‑point advantage over safety framing) across various steering conditions. A chain‑of‑thought pre‑fill intervention further suggests a causal link, with most pre‑fills shifting compliance in the predicted direction, indicating that evaluation awareness is not a uniform behavior but varies qualitatively with its framing.

By Allison Zhuang, Santiago Aranguri
arXiv AI
Aug 28

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic

The paper presents a modeling and simulation framework to study reinforcement learning (RL) control of connected and automated vehicle (CAV) platoon joining maneuvers in mixed traffic. Using SUMO and agent-based modeling, it evaluates Deep Q-Network (DQN), Double DQN (DDQN), and Proximal Policy Optimization (PPO) algorithms, finding that PPO achieves a 98 % joining success rate with less than 1 % collision rate by incorporating risk penalties. The study also shows a trade‑off between safety, joining effectiveness, and decision efficiency, and demonstrates that an external safety controller can prevent collisions but may reduce joining efficiency.

By Biao Yin, Abderrahmane Kasmi, Nadir Farhi
arXiv AI
Aug 28

Distinct Profiles of Run-to-Run Score Reliability and Expert-Panel Alignment Across Four LLM Evaluators of Simulated Japanese-Language AI-to-AI Counseling

The study examined how four large language models (GPT‑5.5, Gemini 3.5 Flash, Claude Opus 4.8, and Fable 5) scored 18 simulated Japanese‑language AI‑to‑AI counseling sessions compared to ratings from 15 human counseling experts. Each model evaluated every transcript three times on four motivational‑interviewing‑informed dimensions and overall quality, consistently giving higher scores for softening sustain talk and overall quality than the expert panel, though the magnitude varied by model. Run‑to‑run reliability (intraclass correlation coefficients ranging from .33 to .96) did not predict closer alignment with expert judgments, and the models’ ability to discriminate counselor conditions was distinct from both reliability and alignment.

By Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Got\=o, Tomonori Hosokawa, Makoto Nishimura, Y\=osuke Sat\=o, Izumi Sezai, Tomohiro Inoue