AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 7

Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation

The paper studies structural priming in language model production by conducting controlled sentence‑completion experiments on dative constructions. Results show that language models exhibit priming effects, especially when sentences are semantically coherent, with stronger relative increases for double‑object datives and larger absolute increases for prepositional‑object datives. The study also finds that primed completions involve more lexico‑semantic repetition, indicating that priming operates across syntactic, lexical, and semantic levels.

By Giulia Pucci, Ruizhe Li, Arabella Sinclair
arXiv AI
Sep 7

ARIA - An Agentic Framework for Autonomous Testing of Infotainment Systems

ARIA is a multi‑agent large‑language‑model framework that autonomously runs end‑to‑end visual tests on Android infotainment systems. From simple scenario sentences, it executes interactions, generates reproducible scripts, and produces detailed reports with visual evidence. In evaluation on a manufacturer’s device, ARIA achieved a 93.3% completion rate, correctly identified all known defects, and demonstrated lower false‑positive rates compared to a single‑agent baseline.

By Ant\'onio Azevedo, Bruno Lima, Jo\~ao Pascoal Faria
arXiv Machine Learning
Sep 7

TNFlow: Amortized Posterior Inference for Trans-Neptunian Object Surface Composition

TNFlow is a transformer‑based normalizing flow model designed to infer the surface composition of Trans‑Neptunian Objects (TNOs) from their reflectance spectra. It is trained on synthetic spectra generated by the Shkuratov radiative transfer model and can invert a spectrum in about 0.7 s on a single CPU core, producing a multimodal posterior over simplex‑valid compositions and grain sizes. On synthetic data, the model’s highest‑weight mode achieves a mean total‑variation distance of 0.149 from ground truth, and it generalizes well to unseen component combinations, though qualitative tests on real JWST spectra reveal potential biases linked to simulator fidelity or training data.

By Agastya Gaur (University of Illinois Urbana-Champaign, SETI Institute), Cristina M. Dalle Ore (Carl Sagan Center, SETI Institute), Alessandra Ricca (NASA Ames Research Center, NASA Ames Research Center)
arXiv Computer Vision
Sep 7

MultiAttenGastro: Multi-Dimensional Attention Augmentation for Gastrointestinal Endoscopy Classification

MultiAttenGastro is a plug‑and‑play attention framework that adds parallel 1‑D channel, 2‑D spatial, and 3‑D contextual heads to existing CNN and transformer backbones for gastrointestinal endoscopy classification. Across eight backbones and five public GI datasets, the framework improves performance on large‑gap datasets such as Kvasir‑Capsule but shows no benefit on small‑gap benchmarks like Kvasir‑v2, with mixed results elsewhere. Analysis using Centered Kernel Alignment indicates that the gains are linked to representational redundancy: low inter‑head redundancy under large domain gaps yields consistent improvements, while high redundancy under small gaps leads to losses.

By Sadhana Devarajan, Praveen Kumar Chandaliya, Dhruvin Jashvant Kumar Shah, Kishor Upla, Kiran Raja
arXiv AI
Sep 7

Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

The paper introduces MedTraj, a framework that constructs, evaluates, and optimizes multi‑step reasoning trajectories for medical AI agents. It parses each trajectory into observations, evidence, numbered steps, and a conclusion, scoring them on coherence, evidence support, hallucination, completeness, and traceability. Experiments on CareQA, PubMedQA, and CECMed show that incorporating quality‑weighted trajectory context improves reasoning coherence and correctness while significantly reducing hallucinations.

By Yunqi Zhu, Wensheng Zhang, Xuebing Yang
arXiv Machine Learning
Sep 7

Hidden In Plain Gaze: Gaze Representations as Privacy Controls for Utility and Re-identification Risk in XR

The paper investigates how different gaze data representations affect privacy and utility in extended reality (XR) systems. Three representations—raw gaze, spatial attention heatmaps, and engineered eye‑movement features—are compared using the HoloAssist dataset, evaluating action recognition accuracy and closed‑set user re‑identification. Engineered features preserve about 85% of action‑recognition performance while reducing re‑identification risk by roughly an order of magnitude, yet still leave some identity leakage, indicating that abstraction alone does not guarantee privacy.

By Cory Ilo, Brendan-David John, Doug A. Bowman
arXiv Machine Learning
Sep 7

FluxDisco: Symbolic Regression for Stoichiometric Dynamical Systems via Monte Carlo Graph Search

FluxDisco is a physics‑informed symbolic regression framework designed for flux‑based, stoichiometric ordinary differential equation systems. By incorporating known stoichiometry, it narrows the search space and guarantees that discovered equations respect physical laws. The method adapts a Monte Carlo Graph Search algorithm to jointly discover fluxes and dynamics, and is evaluated on various physical and biological systems, successfully recovering interpretable governing equations.

By Cassandra Durr (Lancaster University), Alvaro K\"ohn-Luque (University of Oslo), Chris Jewell (Lancaster University), Lloyd A. C. Chapman (Lancaster University)
arXiv Machine Learning
Sep 7

SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

The paper introduces SMILE, a self‑explainable multimodal information bottleneck framework for medical diagnosis. It jointly optimizes predictive accuracy and modality‑specific explainability by selecting the most informative elements within each data modality. Experiments on diverse medical datasets show strong diagnostic performance, including a 9.1‑percentage‑point accuracy gain on the iCTCF dataset, and provide transparent, modality‑aware explanations that enhance both explainability and generalization.

By Yuqing Yang, Alexander Schmatz, Zhaozhao Ma, Changkyu Choi, Robert Jenssen, Shujian Yu
arXiv Computation and Language
Sep 7

LentEx: Generalizable Latent Entity Extraction via Synthetic Data and Instruction-Tuned LLMs

LentEx is a new framework for latent entity extraction that uses synthetic data generation and instruction fine‑tuning to train smaller, efficient large language models. By creating diverse, contextually rich synthetic examples through a template‑based approach, LentEx overcomes the lack of labeled datasets and achieves strong performance, surpassing state‑of‑the‑art models on the MTEB Clustering Benchmark. The method also generalizes well to unseen domains, making it useful for tasks such as retrieval‑augmented generation, customer persona analysis, and knowledge graph enrichment.

By Umesh Bodhwani, Yuan Ling, Cibi Chakravarthy Senthilkumar, Shujing Dong, Yarong Feng, Hongfei Li, Ayush Goyal
arXiv Computation and Language
Sep 7

A Systematic Comparison of Multilingual Interpretability Methods Reveals Anisotropy-Driven Failures

The paper evaluates four metrics—CKA, ANC, GMM dominance per token, and ILO—used to measure cross‑lingual representation sharing in multilingual language models. Across 21 models ranging from 125 M to 14 B parameters, the metrics disagree, and the authors attribute this to anisotropy, where representations cluster in a narrow embedding cone. Only ILO shows a strong, robust correlation with cross‑lingual transfer performance (Spearman’s ρ = 0.90) after controlling for model size, family, and task variation, leading the authors to recommend ILO as the primary metric alongside anisotropy diagnostics.

By Oskar Holmstr\"om, Marcel Bollmann, Marco Kuhlmann
arXiv Computation and Language
Sep 7

On Epistemic Diversity in Large Language Models

The paper introduces the concept of epistemic diversity for large language models (LLMs), defining it as the range of valid answers, explanations, and reasoning routes that an LLM presents to users. It argues that evaluating LLMs solely on accuracy or alignment is insufficient, especially when LLMs are used for knowledge-intensive tasks. The authors propose a preliminary framework for measuring epistemic diversity and demonstrate that leading LLMs often collapse large valid answer spaces into small canonical subsets, indicating epistemic narrowness.

By Elisabeth Kirsten, Nicole Kr\"amer, Muhammad Bilal Zafar
arXiv Machine Learning
Sep 7

Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning

The paper investigates how federated unlearning systems that broadcast updated linear classifiers after each client update can inadvertently leak the compact, additive summaries used for deletion. By submitting known changes and analyzing the returned classifiers, an attacker can recover the deleted sample’s class or even reinstate it. Experiments on MNIST and CIFAR‑10 show that high‑precision broadcasts enable exact label recovery, while lower precision limits fine‑grained recovery and diverse responses can prevent identification.

By Yijun Quan, Giovanni Montana
arXiv Computation and Language
Sep 7

MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering

MedProb is a lightweight probing framework that predicts multiple-choice medical visual question answering (Med‑VQA) answers directly from frozen vision‑language model (VLM) representations, avoiding free‑text generation. On datasets such as PATH‑VQA, SLAKE, and VQA‑RAD, MedProb extracts more answer‑relevant signal than prompting and outperforms both medical VLMs and agentic systems. The approach also narrows the performance gap between small and large models, shows that medical adaptation does not consistently improve linear decodability, and reveals positional biases in both prompting and generation.

By Erfan Nourbakhsh, Ke Yang, Anthony Rios
arXiv AI
Sep 7

From Matching Models to Recruiting Agents: A Systematized Narrative Review of AI Recruitment Systems, Evaluation, and Governance

The paper reviews the evolution of AI recruitment systems from simple profile matching to complex, multi‑stage workflows that retrieve evidence, compare candidates, and execute actions. It analyzes 40 representative works, highlighting transitions from similarity to reciprocal suitability, from single models to compound workflows, and from offline predictions to evidence‑aligned evaluation. The authors identify persistent gaps—such as confounded behavioral labels, limited data validity, and lack of privacy assessment—and propose a staged mapping for defensible evaluation and an agenda for auditable, evidence‑grounded systems.

By Ziyi Zhao, Guanzheng Wei
arXiv AI
Sep 7

Data-Optimized Contingency Screening: A Machine Learning Approach to Power System Security

The paper presents a machine‑learning framework for classifying power‑system contingencies into safe, moderate, or severe categories. Using Newton‑Raphson load flow data, the study applies SMOTE, PCA, and classifiers (KNN, Random Forest, SVM) to IEEE‑14 and IEEE‑30 bus systems, evaluating performance with precision, recall, and F1 score. Random Forest achieved the highest F1 scores, while PCA improved overall performance more than SMOTE, which boosted recall at the cost of some false positives.

By Joshua Salako, Folajimi Osikomaiya, Olakorede Olamiju
arXiv AI
Sep 7

Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection

The paper introduces a training‑free, inference‑time technique to curb hallucinated transcripts in Whisper by projecting decoder activations away from a low‑rank hallucination subspace derived from non‑speech data. Two variants are tested: an always‑on projection that dramatically lowers hallucination rates on non‑speech benchmarks, and a gated version that applies the projection only when non‑speech is predicted, achieving a smaller but still significant reduction. On LibriSpeech, the gated method slightly increases word error rate but keeps false‑rejection rates low, demonstrating a controllable trade‑off between hallucination suppression and recognition accuracy.

By Maryam Abbasihafshejani, Murtuza Jadliwala
arXiv AI
Sep 7

Shadow Queries for Private Retrieval in Vector Databases

The paper introduces SHAQ, a defense called Shadow Query Generation that protects document embeddings in vector databases from embedding inversion attacks. SHAQ replaces direct embeddings with diverse shadow queries generated by a language model, thereby decomposing document semantics and decoupling stored embeddings from the original text. Experiments on various IR datasets show that SHAQ significantly lowers recovery rates, defends more tokens than baseline methods, and even improves retrieval utility.

By Xinguo Feng, Zhongkui Ma, Zihan Wang, Chuan Yan, Guowei Yang, Alsharif Abuadbba, Guangdong Bai
arXiv AI
Sep 7

When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models

The paper investigates numerical hallucination in financial language models, comparing a base instruction-tuned model, a domain-adapted model (FT‑A), and a numeracy-enhanced domain model (FT‑A+B+C). A three-level detectability taxonomy is introduced, and the study finds that domain fine-tuning dramatically increases hallucination rates—up to 98%—and that numeracy supervision actually amplifies hallucination. Template injection is identified as a key mechanism, leading the authors to conclude that numerical hallucination stems from degraded numerical restraint rather than lack of reasoning, and they recommend comprehensive evaluation and grounding-aware deployment practices.

By Xiaodong Li, Peiwei Liu
arXiv AI
Sep 7

MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models

MM-IFEval-Pro is a new multilingual benchmark for evaluating instruction-following in vision-language models, covering both Chinese and English tasks. It includes 4 major task categories, 24 subcategories, and 8 instruction categories with 52 subcategories, each sample featuring an average of 3.0 constraints to mimic complex instruction scenarios. A reinforcement-learning training set with Chinese and adversarial instructions improves model performance on MM-IFEval-Pro and transfers well to other multimodal benchmarks, showing strong cross-task and cross-language generalization.

By Changming Xiao, Zhenliang Ni, Jinhui He, Han Shu, Jie Hu