AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv AI
Aug 20

GenEx: A Graph-Based Representational Paradigm for SARS-CoV-2 Variant Detection via Codon Co-occurrence Networks

GenEx is a new pipeline that transforms raw SARS‑CoV‑2 gene sequences into codon co‑occurrence graphs, extracting over 25 graph features. It introduces two graph generation methods—MSCG and LAPCG—and applies spectral graph analysis via Singular Value Decomposition to improve class separability. Using these techniques, the authors trained 23 machine‑learning models that successfully detected all major SARS‑CoV‑2 variants.

By Arefin Amin, Labiba Faiza Karim, M. Monir Uddin
arXiv Machine Learning
Aug 20

SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection

The paper introduces Structured Prior Knowledge (SPK), a framework that extracts and organizes latent priors from pretrained object detectors to improve out-of-distribution (OoD) detection. SPK uses in-distribution data and hallucination-inducing samples to elicit part-level semantic concepts, then combines these with geometric and contextual priors into a compact five-dimensional representation. Experiments across various detector architectures and OoD benchmarks show that SPK achieves state-of-the-art performance, demonstrating that pretrained detectors encode richer latent knowledge than previously exploited.

By Changshun Wu, Weicheng He, Xiaowei Huang, Saddek Bensalem
arXiv Machine Learning
Aug 20

Simple, Safe, and Overlooked: Reclaiming Sustainable Domain Generalization with Statistical Color Matching

The paper introduces Colorist, a data‑augmentation method that uses classical statistical color matching to generate domain‑shifted medical images. By applying global mean‑standard‑deviation matching in RGB space, Colorist creates structurally intact variations without neural networks, outperforming deep generative models in fidelity and color alignment. Across multiple medical imaging datasets, it boosts balanced accuracy by up to 9% over state‑of‑the‑art domain‑generalization regularizers and 13% over no augmentation, while reducing computational cost and preserving anatomical structure.

By Sebastian Doerrich, Francesco Di Salvo, Shyam Nandan Rai, Marco Lents, Christian Ledig
arXiv AI
Aug 20

Breaking the weakest link to evade vision language models

The paper investigates how Vision Language Models (VLMs) can be fooled by small, human‑imperceptible changes to images. It introduces a gradient‑based attack that targets only the vision encoder, reducing computational cost while still effectively disrupting both untargeted and targeted multimodal alignment. Experiments on open‑source VLMs such as Qwen2.5‑VL, Granite‑Vision, FastVLM, and Phi‑3.5‑Vision demonstrate that these perturbations can significantly alter the models’ textual outputs.

By Ilan Zini, Boussad Addad, Katarzyna Kapusta
arXiv AI
Aug 20

Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

The paper introduces Latent Space Refusal Anchoring (LSR‑Anchoring), a training‑free technique that extracts a refusal direction from English prompts and applies it to the residual stream of instruction‑tuned models at inference time. The primary variant, Mean‑Activation Steering (MAS), works across several architectures (Llama‑3‑8B, Llama‑3.1‑70B, Mistral‑7B‑Instruct, Qwen2.5‑7B), restoring safety for low‑resource African languages with minimal performance loss, while a refined SAE‑Derived Steering (SDS) further reduces KL divergence without degrading legitimate prompt performance. The method shows positive transfer for Yoruba, Igbo, Igala, and Hausa, but fails for Arabic, suggesting a geometric mismatch rather than a data scarcity issue.

By Godwin Abuh Faruna
arXiv AI
Aug 20

Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems

Fractional Decay KV-Cache (FD‑KVC) is a new memory‑management algorithm for transformer‑based dialog systems that uses a dual‑channel scoring mechanism—cumulative attention and recency‑weighted relevance—to keep key‑value pairs relevant as conversation topics shift. The method incorporates an ownership‑loss‑driven adaptive learning rate, runs entirely on CPU with minimal overhead, and outperforms the heavy‑hitter baseline H2O by significant margins across multiple dialog scenarios, including faster topic adaptation and higher topic diversity. Ablation studies confirm that each component of FD‑KVC contributes to its performance gains.

By Sukanta Ganguly
arXiv AI
Aug 20

Computational Orientalism: Measuring Structural Discourse Bias in Large Language Models Using the Middle East Cultural Sensitivity Score (MECSS)

The paper introduces the Middle East Cultural Sensitivity Score (MECSS) to quantify Orientalist bias in large language models, converting Said’s seven Orientalist operations into measurable dimensions. Using 280 conversations, it finds that GPT‑4 and Falcon3‑7B‑Instruct systematically reproduce Orientalist patterns, with Falcon scoring higher despite being regionally built. The study highlights that geographic origin alone does not mitigate bias and identifies a new failure mode, "Said‑washing," present in 87.9% of GPT‑4 interactions.

By Maha Shahid
arXiv Machine Learning
Aug 20

MITRE-SAGE: A Multi-Agent Cybersecurity Question-Answering Model

MITRE‑SAGE is a multi‑agent retrieval‑augmented generation framework that combines semantic and structural cybersecurity knowledge to enhance large language model question‑answering. It decomposes tasks into query interpretation, evidence retrieval, and answer synthesis, supporting vulnerability assessment, threat profiling, and relationship extraction. The authors also introduce MITRE‑QA, a benchmark of 3,000 question‑answer pairs, and show that MITRE‑SAGE outperforms standalone LLMs and conventional RAG methods, with a lightweight configuration achieving top performance on most tasks.

By Ali Habibzadeh, Farid Feyzi, Reza Ebrahimi Atani
arXiv AI
Aug 20

FairGlucose: A CGM Fairness Benchmark Reveals Subgroup Disparities Hidden in Population-Level Validation

FairGlucose is a 300‑patient CGM cohort balanced across 12 demographic strata, providing 132,480 forecasting samples and 3,945 behavioral events. Benchmarking 33 models on 2‑hour glucose forecasting revealed that population‑level validation masks significant subgroup disparities, with error ratios ranging from 0.8 to 1.4 and T1D patients experiencing 6 mg/dL higher error than T2D. The study shows that these gaps persist across all models, align with clinically hard cases, and vary with input‑length sensitivity, underscoring the need for subgroup‑disaggregated reporting in digital health AI.

By Junjie Luo, Xuzhe Zhi, Rui Han, Abhimanyu Kumbara, Anand K. Iyer, Mansur E. Shomali, Ritu Agarwal, Guodong Gordon Gao
arXiv AI
Aug 20

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model

arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.

By Qi Yu, Zhichen Zeng, Katherine Tieu, Xiyuan Yang, Ruizhong Qiu, Yuchen Yan, Lihui Liu, Yanjun Zhao, Lingjie Chen, Jingrui He, Hanghang Tong
arXiv AI
Aug 20

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

The paper investigates the vulnerability of Gaussian‑noised text embeddings to inversion attacks. It identifies a "Double Noise Trap" that hampers standard generative methods and introduces DAEI, a denoising‑aware pipeline that significantly outperforms existing baselines in reconstructing original text. Experiments show DAEI improves BLEU by 154% and token‑level metrics by 32–60%.

By Yubo Wang, Shujie Cui, James Bailey, Hongzhi Yin, Wenyu Liang, Min Tang, Shiyue Qin, Weiqing Wang
arXiv AI
Aug 20

Decomposing Wrong-Consensus Agreement in LLM Self-Consistency: A GPT-4.1 Case Study

The paper introduces a pluralistic agreement index, Gamma, to quantify how often wrong runs of large language models (LLMs) agree with the majority consensus. By decomposing Gamma into a mechanical component and a preference‑unexplained residual, the authors show that on GPT‑4.1 the mechanical part explains most of the agreement on multiple‑choice benchmarks but only about half on open‑domain tasks, revealing a residual bias that can cause self‑consistency to backfire on hard questions. The study provides a quantitative framework for understanding when majority voting over LLM samples improves or harms accuracy, without proposing new voting methods.

By Lizhuo Zhang, Mengmeng Tang, Chenfeng Long, Xiaoyong Tang, Xiang Luo
arXiv AI
Aug 20

Do Large Language Models Hallucinate Electric Fata Morganas?

The paper investigates why large language models (LLMs) produce hallucinations—outputs that are fabricated, unverifiable, or contradictory to source material—and argues that these hallucinations have philosophical implications for machine consciousness. It reviews known causes such as source‑target divergence, training‑inference discrepancies, and overfitting, and presents two empirical studies: one showing that higher temperature settings in GPT models yield plausible but incorrect answers, while lower temperatures produce accurate ones; and another demonstrating that an encoder‑only model trained on encyclopedic data answers factually without embellishment, suggesting hallucinations arise from exposure to subjective, socially diverse data rather than cognitive ability. Drawing on Turing, Searle’s Chinese Room, the frame problem, and cybernetic theory, the authors contend that a model’s self‑reports of emotion or sentience fall within the definition of hallucination, implying that any future machine consciousness may remain epistemically inaccessible because it would be indistinguishable from an advanced hallucination.

By Kristina \v{S}ekrst
arXiv AI
Aug 20

Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead

The article reviews the growing use of Large Language Models (LLMs) for generating Verilog code, a key hardware description language in electronic design automation. It surveys 102 papers, covering conferences, journals, and preprints, and addresses four research questions about LLM selection, datasets, techniques, and alignment strategies. The review identifies current limitations and proposes a roadmap for future research in LLM-assisted hardware design.

By Guang Yang, Wei Zheng, Xiang Chen, Dong Liang, Peng Hu, Yukui Yang, Shaohang Peng, Zhenghan Li, Jiahui Feng, Xiao Wei, Kexin Sun, Deyuan Ma, Haotian Cheng, Yiheng Shen, Xing Hu, Terry Yue Zhuo, David Lo
arXiv Machine Learning
Aug 20

FedLNS: Leverage LayerNorm Signature Modeling to Mitigate Adversarial Manipulation in Federated LLMs

FedLNS is a server‑side framework that screens federated learning updates by representing each client’s contribution through changes in trainable normalization‑layer parameters, creating lightweight signatures that can be compared against a history‑aware cross‑client reference. The method requires no extra client‑to‑server communication, raw data, or labeled attack examples, and after screening, the remaining full‑model updates are aggregated with standard federated learning rules. Experiments on GPT‑style, BERT‑style, and LLaMA‑style models with 200 clients demonstrate that FedLNS achieves lower test perplexity than six baselines even when 40% of the population performs target manipulation under both IID and non‑IID data partitions.

By Kai Li, Jong-Ik Park, Carlee Joe-Wong, Wei Ni, Falko Dressler
arXiv Machine Learning
Aug 20

Pretraining Reusable Inference Across Views with Synthetic Task Priors

The paper introduces SIMPLE, a prior‑fitted multi‑view in‑context learner that learns a reusable, task‑conditioned inference procedure instead of a fixed fusion function. By generating synthetic task priors in embedding space, SIMPLE can handle diverse view configurations, class structures, and missingness patterns. Experiments on multi‑view and multi‑omics benchmarks show that a frozen SIMPLE model performs competitively, and lightweight adapter calibration further improves performance across most datasets.

By Jielong Lu, Zhihao Wu, Jiajun Yu, Zhaoliang Chen, Haishuai Wang
arXiv Machine Learning
Aug 20

Enhancing EBSD throughput of battery electrode materials using super-resolution generative adversarial networks

The paper presents a super‑resolution generative adversarial network (SRGAN) that boosts electron backscatter diffraction (EBSD) throughput for lithium‑ion battery electrode materials. Trained on LiNixMnyCozO2 cathode data, the SRGAN outperforms classical interpolation across 2×–12× upscaling, especially preserving small grains and realistic boundaries. A 5× upscaling yields a 25× speed‑up or larger field of view with acceptable errors in grain size and shape metrics.

By John Mangum, Andrew Glaws, Francois Usseglio-Viretta, Steven Spurgeon, Donal Finegan
arXiv Machine Learning
Aug 20

Learning Topological Features of $\widehat Z$-invariants

arXiv:2608. 18570v1 Announce Type: cross Abstract: Machine learning and data analysis techniques have recently emerged as powerful tools for identifying patterns and formulating conjectures in mathematical research, most notably in the field of low-dimensional topology.

By Brandon Robinson, Shimal Harichurn, Fabian Ruehle, Sergei Gukov, Rak-Kyeong Seong, Miranda C. N. Cheng
arXiv Machine Learning
Aug 20

Gradient Mirage: Trainable yet Label-Unidentifiable Gradients in Large Language Model Split Learning

Gradient Mirage is a defense for large language model split learning that disrupts the assumption that exposed gradients faithfully reflect the client’s full-label training objective, thereby thwarting gradient matching attacks. It introduces inconsistencies across objective, direction, and scale by using selective autoregressive supervision, scale blinding, and directional privatization with a von Mises-Fisher mechanism. The method preserves utility through dual-track backpropagation and bottom-gradient recovery, achieving a stronger privacy‑utility trade‑off in experiments.

By Shiyu Miao, Yunlong Mao, Zirui Huang, Liang Yao, Tianshuo Zheng, Yanhui Gu, Fan Liu, Sheng Zhong
arXiv Machine Learning
Aug 20

MIFR: A Modality-Invariant and Fair Representation Framework for Skin Disease Classification

The paper introduces MIFR, a modality‑invariant and fair representation framework for skin disease classification that jointly processes clinical photographs and dermoscopic images using ViT‑based encoders. It employs a five‑component multi‑objective loss to balance classification accuracy, fairness across skin tones, class alignment, and modality invariance. Experiments on paired and external datasets demonstrate competitive predictive performance and fairness, with t‑SNE visualizations confirming alignment of embeddings from different modalities.

By Asonyu Senge Njih, Yvan Guifo Fodjo, Vianney Kengne Tchendji, Jerry Lacmou Zeutouo, Kerol Djoumessi