AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Computer Vision
Aug 27

Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark

The paper introduces ACF-Net, an optical flow‑guided framework for asymmetric audio‑visual fine‑grained visual categorization (FGVC), addressing challenges where video and audio are not strictly synchronized or matched. ACF-Net comprises Optical Flow‑Guided Motion (OFGM) to capture motion‑sensitive visual cues and suppress background noise, and Asymmetric Cross‑Modal Adaptive Fusion (ACAF) to estimate modality reliability and perform uncertainty‑aware fusion. The authors also present BirdPro, a new bird‑oriented audio‑visual benchmark with 1,919 audio recordings and 11,965 videos across 194 species, and report that ACF‑Net outperforms baselines by 2.97% in fused and 1.92% in mismatched settings.

By Bohan Deng, Shuo Ye, Zitong Yu
arXiv Computer Vision
Aug 27

GraftSR: Grafting Authentic Textures for Real-World Image Super-Resolution via Identical-Instance Guidance

GraftSR is a diffusion-based super‑resolution framework that uses reference images of the same object to guide texture restoration, mitigating hallucination. It introduces a dual‑mask reference guidance mechanism to decouple texture extraction from application, avoiding reliance on spatial alignment. The authors also release TexRefSR‑141K, a large dataset of reference pairs with spatial masks, and show that GraftSR outperforms existing methods on the TexRefSR‑Eval benchmark, reducing LPIPS by 20.2%.

By Qifan Yu, Haoran Bai, Zongyao He, Weijie He, Sibin Deng, Honggang Qi, Ying Chen
arXiv Machine Learning
Aug 27

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure

The paper critiques a recent NLI benchmark that tests the imperfective paradox, arguing that the benchmark suffers from conceptual and evaluation mis-specifications, notably Aspectual Reduction and a lack of strict NLI standards. The authors re-evaluate the benchmark, identify mis-specifications, and construct lexically matched minimal pairs to control for lexical variation. Their experiments reveal that models often exhibit a Sufficiency Bias, accept simple‑past hypotheses without affirming culmination, and that prompting interventions shift label decisions without improving true semantic understanding, highlighting additional failure modes such as compositional aspectual classification errors and surface‑form attraction.

By Kaiqiao Han, Yizhou Sun
arXiv Machine Learning
Aug 27

PA-CoT: Profile-Adaptive Chain-of-Thought for Personalized Nutritional Consulting

PA-CoT (Profile-Adaptive Chain-of-Thought) is a multi-stage prompting method that explicitly analyzes user profiles before generating responses for personalized nutritional consulting. The authors introduce the QPA benchmark, comprising 200 samples with structured profiles scored on four criteria, to evaluate personalization and safety. In a comparative study against 11 other methods, PA-CoT achieves the highest average score (4.21) and leads in both personalization (4.71) and safety (4.68) with statistically significant margins.

By Evgenii Garmashov, Nikita Kulin, Artur Khairullin, Viktor Zhuravlev, Daniil Sukhorukov, Mikhail Mozikov, Ilya Makarov, Sergey Muravyov
arXiv Machine Learning
Aug 27

Learning Continuous Regional Temperature Fields with Lead-Time and Resolution Queries

The paper introduces the Continuous Spatiotemporal Temperature Forecaster (CSTF), a neural field that predicts 2‑meter temperature (T2M) by treating forecast lead time and output resolution as explicit queries. CSTF encodes ERA5 history into latent states and decodes T2M as a coordinate‑based field, allowing flexible evaluation at any spatial location, lead time, or resolution. Experiments on a Southeast China benchmark show CSTF outperforms existing methods, achieving a 17.0 % bias reduction and demonstrating coherent predictions across varying lead times and resolutions.

By Chunlei Shi, Jiong Wang, Yi-Lin Wei, Junming Hou, Jinjin Liu, Yecheng Zhang, Dan Niu
arXiv Computation and Language
Aug 27

Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study

The paper compares encoder‑based and generative decoder‑based large language models for evaluating automatic speech recognition (ASR). It examines BERTScore and SemDist across various LLMs, layers, and pooling strategies, finding that both metrics can strongly correlate with human judgments when properly configured. For generative LLMs, the study explores pairwise hypothesis selection via prompting and direct error classification, showing that while encoder‑based metrics remain competitive, generative models excel in hypothesis comparison and enhance interpretability of ASR evaluation.

By Thibault Ba\~neras-Roux, Shashi Kumar, Driss Khalil, Sergio Burdisso, Petr Motlicek, Shiran Liu, Mickael Rouvier, Jane Wottawa, Richard Dufour
arXiv Computation and Language
Aug 27

EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports

EgoArgus is a new, human‑annotated dataset that tests visual‑language models (VLMs) as situational assistants in five everyday dialogue‑video scenarios. It evaluates how well VLMs understand and decide when to intervene, especially when visual and textual cues are helpful, irrelevant, or conflicting. The study finds that current VLMs still struggle to reliably act as egocentric assistants and that existing modality‑bias mitigation methods offer limited improvement.

By Yu-Chien Tang, Yu-Hsiang Liu, An-Zi Yen
arXiv Computation and Language
Aug 27

ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains

ReliableRAG is a new framework for Retrieval-Augmented Generation that tackles misinformation in multi‑hop question answering. It extracts structured triples from retrieved documents, evaluates each triple’s reliability by combining semantic relevance to the query with credibility, and keeps only the top‑K reliable, non‑redundant triples. Using these refined triples, the system builds robust reasoning chains that filter out deceptive misinformation and produce accurate, trustworthy answers.

By Jinpu Jiang, Xuan Wu, Wenhao Song, Bo Yang, You Zhou, Hongwei Ge, Heow Pueh Lee, Yanchun Liang, Chunguo Wu
arXiv Computation and Language
Aug 27

Adaptive Triggering for Bias Correction in LLM Reasoning

The paper introduces an adaptive triggering mechanism for bias correction in large language model (LLM) reasoning. By framing bias intervention as an online change‑point detection problem, the authors update a CUSUM statistic at each step using either a white‑box next‑token probability signal or a black‑box LLM judge signal, and inject corrective prompts only when the accumulated evidence exceeds a calibrated threshold. Experiments on gpt‑4o‑mini and six open‑weight models show that adaptive black‑box triggering restores most of the accuracy lost by fixed‑interval interventions while reducing the number of corrections, whereas the white‑box signal improves ambiguous‑item accuracy but can hurt disambiguated‑item accuracy due to difficulty distinguishing stereotype reliance from correct evidence.

By Nayoung Kim, Mickey Mancenido, Huan Liu
arXiv Machine Learning
Aug 27

Behind the [MASK]: Disentangling Representation and Faithfulness in DAPF-Based Dementia Detection

The paper investigates the interpretability of Domain-Adapted Prompt-based Fine-tuning (DAPF) models for dementia detection from spoken language. Using probing and analysis techniques, the authors find that DAPF achieves strong overall performance (accuracy = 0.83, macro‑F1 = 0.83) and that the diagnosis is most recoverable from its [MASK] representation. However, token‑level explanations derived from DAPF are largely driven by language‑task vocabulary, discourse markers, and transcription artifacts, and perturbation tests reveal weak or negative effects, indicating that the masked‑token interface captures diagnosis information without providing faithful token‑level explanations.

By Pardis Ranjbar-Noiey, Natalie Parde
arXiv Machine Learning
Aug 27

Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation

The paper surveys attacks and defenses in Retrieval-Augmented Generation (RAG), a technique that improves large language models by grounding outputs in external knowledge. It identifies new robustness and security risks such as corpus poisoning, backdoor attacks, privacy leakage, and fairness violations, and notes that existing surveys inadequately cover attacker objectives, threat models, and stage-specific defenses. The survey offers a unified, pipeline-aware overview, formalizing threat models across the corpus, retriever, and generator, categorizing attacks by accuracy, privacy, and fairness, and reviewing defenses for retrieval, rerank, generation, and traceback stages, while also summarizing robustness benchmarks and explainability methods.

By Minh Tran, Cuong Dang, Tuc Nguyen, Khanh-Tung Tran, Minh Huynh Nguyen, Trinh Chau, Kien Le, Do Xuan Long, Jiahao Zhang, Hoang D. Nguyen, Thanh Le, Suhang Wang
arXiv Machine Learning
Aug 27

GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization

GRAPE is a two‑stage Bayesian optimization framework that first refines the local gradient posterior using a closed‑form acquisition function and then selects update directions by maximizing expected decrease conditioned on descent. The authors prove that the refinement stage monotonically reduces local uncertainty and that the progress‑aware direction converges to true steepest descent as the posterior sharpens. Empirical results show GRAPE achieves a 5.4× speedup on black‑box adversarial attacks and reduces final average regret by 3.8 log‑units on large language model prompt‑optimization tasks.

By Richard Cornelius Suwandi, Feng Yin
arXiv Machine Learning
Aug 27

Adversarial Training of Linear Models under Stealthy Attacks

The paper introduces a detector‑based switched model to defend linear predictive models against stealthy false data injection attacks. It derives a convex formulation of the adversarial risk that incorporates protected features and a hyperparameter for attack probability, allowing an explicit trade‑off between clean and attacked data performance. Numerical experiments on real and synthetic datasets demonstrate improved performance on partially attacked data, even when the attack probability is misspecified.

By Lovisa Eriksson, Dave Zachariah, Andr\'e M. H. Teixeira
arXiv Machine Learning
Aug 27

When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study

The paper investigates when frequency decomposition aids Physics-Informed Neural Networks (PINNs) by introducing a dual‑branch, spectrally‑gated architecture (DBSG‑PINN) that separates low‑ and high‑frequency components. Experiments on five one‑dimensional PDE benchmarks show that frequency decomposition significantly reduces error—up to 59.2% on a multimodal wave problem—when the target solution is spectrally complex, but offers little improvement on smoother problems and can even worsen performance on a simple 1D wave benchmark. The adaptive gate’s effectiveness scales with the spectral richness of the solution, suggesting it exploits frequency structure rather than adding noise.

By Shubham Rai
arXiv Machine Learning
Aug 27

AERIS: Offline Policy Improvement for Multi-UAV Integrated Sensing and Communication

AERIS is an offline policy improvement framework for multi-UAV integrated sensing and communication (ISAC) that learns from fixed flight logs using centralized training and decentralized execution. It introduces STAR-CRDT, an offline multi-agent RL algorithm that rectifies local actions and distills trusted improvements into decentralized actors, providing an offline-support policy improvement guarantee. Experiments demonstrate that STAR-CRDT boosts the main ISAC objective return by 29.3% and improves communication sum rate, sensing pass rate, and sensing margin while reducing collision-risk events by 54.2%.

By Ziyuan Wang (Steven), Yifan Sui (Steven), Wei Wei (Steven), Wenjie Xin (Steven), Zekai Zhang (Steven), Xiangwang Hou (Steven), Xiao-Ping (Steven), Zhang
arXiv Machine Learning
Aug 27

Controlling for Omitted Variable Bias in Deep Neural Networks

The paper introduces a control‑variable framework for deep neural networks to mitigate omitted variable bias, particularly shortcut learning where covariates like demographics influence predictions. It refits the final layer of a pre‑trained network using cross‑fitting with ridge penalisation, orthogonalises covariate effects, and marginalises predictions over covariate distributions to achieve unbiased, interpretable results. Experiments on simulated images and neuroimaging data show consistent estimation of true effects and performance close to models trained on unconfounded data.

By Manuel Pfeuffer, Roshan Prakash Rane, Kerstin Ritter, Sonja Greven
arXiv Machine Learning
Aug 27

Quantum-Inspired Modeling of Driving Behavior

The paper introduces a quantum-inspired representation of driver behavior that models drivers as evolving density matrices, capturing continuous, probabilistic, context-dependent, and history-dependent interactions among behavioral variables. Trained unsupervised on the I‑24 MOTION dataset, the framework identifies three interpretable driving regimes—free flow, transition, and congestion—and reproduces macroscopic traffic phenomena such as the fundamental diagram and hysteresis loops. The representation also enhances practical applications by providing context-dependent parameters for classical car‑following models and enabling autonomous vehicles to forecast nearby drivers’ motion in real time.

By Mohammad Elayan, Omid Armantalab, Wissam Kontar
arXiv Machine Learning
Aug 27

Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment

The paper introduces MoPLEx, an expectation‑maximization algorithm for learning mixtures of Plackett‑Luce models from multi‑way ranking data. It augments rankings with synthetic responses from a base language model and uses a gradient‑based estimation to reduce inference cost, enabling efficient fitting of large‑scale models. Experiments show the method achieves low probability estimation error and improves clustering and ranking accuracy by 43.7% and 15.2% over baselines.

By Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang
arXiv Machine Learning
Aug 27

DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search

The paper introduces DCEO, a data‑driven framework that learns item‑level proxy scores directly aligned with long‑term user objectives in e‑commerce search. It aggregates these scores into a user‑level metric, measures alignment via relative causal effect, and uses an actor‑critic model to generate context‑dependent fusion weights for multiple objectives. Offline experiments and a 41‑day online A/B test show DCEO improves GMV by 0.36% over traditional proxies.

By Junzhao Zhang, Tao Zhang, Liren Yu, Feiyi Dong, Zhixuan Zhang, Dan Ou, Haihong Tang