AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Sep 22

What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior

arXiv:2609.24576v1 Announce Type: cross Abstract: Modern Vision-Language Navigation (VLN) models rely mostly on pre-trained large Vision-Language Models (VLMs) to predict navigation actions. While th...

By D\'ebora Oliveira Makowski, Samiran Gode, Abhijeet Nayak, Marco Hutter, Cordelia Schmid, Lukas Rosenberger Schmid, Wolfram Burgard
arXiv Machine Learning
Sep 22

When Does Adversarial Refinement Help? A Negative Result and Open Problem in Adapting R3GAN to Time Series Imputation

The paper investigates whether the stable GAN architecture R3GAN can improve time‑series imputation when adapted to 1‑D temporal data. Using a coarse‑to‑fine refinement framework and a frequency‑domain discriminator, the authors evaluate 14 saved configurations across three datasets and find a negative result: most configurations either show negligible improvement or degrade performance compared to baseline methods. The study highlights that the usual argument—GANs optimize distributional objectives rather than point‑wise ones—does not fully explain the lack of benefit, and it poses an open problem regarding why a learned discriminator fails to provide useful refinement gradients while diffusion denoisers succeed, offering practical guidance on when adversarial refinement may be worthwhile.

By Yufeng He
arXiv Machine Learning
Sep 22

Adaptive Forgetting for Nonstationary Optimization: Towards Robust EEG Decoding

The paper introduces AFOR, a tensor‑wise adaptive optimizer for EEG decoding that replaces the fixed second‑moment decay coefficient used in Adam/AdamW with a dynamic coefficient estimated online from local gradient state. AFOR combines a Residual‑Alignment Signal Scorer (RASS) to assess gradient quality and an Adaptive Forgetting Controller (AFC) to map this score to a bounded per‑step decay coefficient, with cumulative‑product initialization correction for consistency. In cross‑subject experiments on three EEG benchmarks, AFOR outperforms standard optimizers, improving mean test accuracy by 3.00%, 2.07%, and 4.38% over Adam.

By Hongyu Zhu, Lin Chen, Jing Chen, Yuting Zhou, Mingsheng Shang
arXiv Machine Learning
Sep 22

Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders

The paper compares latent representations in Selective State Space Models (SSMs) like Mamba and Transformers such as Pythia using Sparse Autoencoders. Across a 10‑million token corpus, 99.98% of Mamba features align closely with Pythia’s, supporting the Universality Hypothesis that core semantic representations are similar across architectures. A tiny 0.02% of features diverge, with Mamba’s recurrent bottleneck causing it to compress syntactic anomalies into polysemantic neurons, whereas Pythia’s attention can isolate distinct formatting edge‑cases.

By Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder, Ashwini M Joshi
arXiv Machine Learning
Sep 22

Graph Learning for Cross-Subject, Cross-Population EEG Emotion Decoding and Model-Derived Spatial-Spectral Neural Signatures

The paper introduces EmoDiPyraTrans, a development-regularized differential graph Transformer designed to decode emotions from EEG signals across unseen individuals and populations while maintaining neural interpretability. Evaluations on five datasets (SEED, FACED, MAHNOB-HCI, DEAP, DREAMER) show high cross‑subject accuracies, with the model outperforming others in accuracy and positive‑class F1. Additional experiments demonstrate its ability to distinguish healthy from depressed participants and reveal spatial‑spectral neural signatures across frontal, temporal, central, and parietal regions with an alpha‑centered frequency preference.

By Dongyi He, Bin Jiang, Xiangkai Wang, Yun Zhao, Hongjie Yan, Wai Ting Siok, Nizhuan Wang
arXiv Machine Learning
Sep 22

Fairness Beyond Anonymization? Demographic Leakage in German LLM-Generated Resumes

The study audits demographic leakage in German-language resumes generated by large language models. Using ChatGPT, Gemini, and Qwen 3 variants, the authors generate resumes from anonymized profiles, varying only gender- and ethnicity-associated names while keeping qualifications constant. Even after anonymization and gender-neutralization, classifiers can reliably distinguish male- from female-generated resumes, driven by subtle differences in gender-neutral terminology rather than overtly gendered wording; ethnicity-related leakage remains weak.

By Charlotte Leininger, Helena Veit, Matthias A{\ss}enmacher, Andreas Bender
arXiv Machine Learning
Sep 22

Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks

The paper demonstrates that quantum transformer blocks can be intrinsically interpretable by tracking quantum mutual information, entanglement entropy, and state fidelity across layers. Experiments on four synthetic tasks show that learned mutual information aligns with task structure, entanglement is essential for accuracy, and mutual information predicts prediction correctness. These findings are validated on IBM Quantum hardware, illustrating that quantum computation’s physics can provide observable interpretability signals.

By Diego Iacopetta, Andrea Gasparini
arXiv Computation and Language
Sep 22

Analyzing Public Discourse on Urbanism: Topic Clustering, Sentiment Analysis and Retrieval-Augmented Generation using YouTube Comments

The paper introduces a pipeline and conversational system that processes 22,788 YouTube transcript and comment chunks from 309 North American cities to analyze public discourse on urbanism. It combines geographic entity resolution, topic modeling, sentiment analysis, and Retrieval-Augmented Generation (RAG), and reports empirical findings on model performance, such as a Twitter-tuned RoBERTa classifier outperforming VADER and dense retrieval surpassing TF‑IDF. The study also evaluates groundedness metrics, noting limitations of BERTScore and ROUGE‑1 for short user-generated text.

By Jakob Morales, Monica Hegde, Fayeq Jeelani Syed
arXiv Computation and Language
Sep 22

Euston: Training Away Mathematical Sycophancy Without Losing the Mathematics

Euston is an 8‑B parameter mathematical claim‑verification model that resists producing false derivations when presented with corrupted theorems. It was trained on 3,026 matched true/corrupted statement pairs generated by GraphSynth, a probabilistic factor‑graph generator, and fine‑tuned from DeepSeek‑R1‑8B using GRPO. On a balanced held‑out split, Euston’s balanced accuracy rose from 29.50 % to 63.75 %, and its discrimination gap improved from –0.5 % to +27.5 %, while maintaining comparable general mathematical ability and reducing response length and truncation rates.

By Zehua Cheng, Wei Dai, Jiahao Sun
arXiv Computer Vision
Sep 22

Brain-to-Image Generation: Reconstructing Visual Stimuli from EEG using Generative Adversarial Networks

The paper presents a reproducible single‑subject baseline for reconstructing visual stimuli from EEG using a temporal‑spatial convolutional encoder that maps averaged EEG signals to 512‑dimensional ViT-B/32 image features. On the THINGS‑EEG2 dataset, the model achieves 12.83%, 39.17%, and 58.00% image recall at ranks 1, 5, and 10, respectively, outperforming analytical chance levels. The study also shows that performance drops sharply when applying a model trained on one subject to others, and that direct conditional generators without external visual weights produce noise‑dominated outputs, indicating that only coarse semantic decoding is feasible under the tested protocol.

By Harshit Goyal