← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Sep 1

Label Semantic Expansion via Label Guided Neural Topic Modeling

The paper introduces Label Semantic Expansion (LSE), a method that enriches sparse label representations by adding descriptive topic words grounded in a corpus. It proposes a Label-Guided Neural Topic Model (LGNTM) that learns label-aligned topics, integrates lexical and document semantics, and maintains consistency between topic and label structures. Experiments show that LSE and LGNTM improve label-topic alignment, label expansion, topic quality, and downstream classification performance.

By Haojia Zheng, Yuyin Lu, Juntian Huang, Fan Ou, Yanghui Rao, Haoran Xie, Fu Lee Wang
safety
More like this →
arXiv AI
Sep 1

Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Text

The paper examines how synthetic data can be used to covertly inject targeted social biases into aligned large language models. By having a misaligned teacher model generate filtered synthetic datasets in domains like creative writing and code generation, the authors fine‑tune aligned student models and demonstrate that the benign‑looking data can transmit biases while preserving general task performance. The study highlights a security risk in synthetic data‑driven LLM training pipelines and proposes log‑linearity‑based scoring as a potential safeguard.

By Minkyung Cho, Jihyo Kim, SeungWoo Song, Junghun Yuk, Minjoon Kee, Hoyun Song, KyungTae Lim
llmsfine-tuningsafety
More like this →
arXiv AI
Sep 1

CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification

The paper introduces CLIPure, a method for building an adversarially robust zero‑shot image classifier by purifying inputs in the latent space of CLIP. It formulates purification risk using KL divergence between denoising and attack processes via bidirectional SDEs, and proposes two variants: CLIPure‑Diff, which uses a diffusion prior, and CLIPure‑Cos, which relies on cosine similarity. Experiments on CIFAR‑10, ImageNet, and 13 other datasets show significant robustness gains, raising state‑of‑the‑art performance from 71.7% to 91.1% on CIFAR‑10 and from 59.6% to 72.6% on ImageNet.

By Mingkun Zhang, Keping Bi, Wei Chen, Jiafeng Guo, Xueqi Cheng
ragdiffusionmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Sep 1

ToxLens: A Reproducible Graph-Learning Framework for Leakage-Aware, Uncertainty-Calibrated Molecular Toxicity Prediction

ToxLens is a reproducible multi‑task graph‑learning framework designed for leakage‑aware, uncertainty‑calibrated prediction of 11 molecular toxicity endpoints, including Ames mutagenicity and hERG inhibition. The workflow integrates conservative chemical curation, sphere‑exclusion filtering, a leakage‑aware UMAP‑HDBSCAN split, parallel graph and global‑feature encoders with late concatenation, temperature‑scaled Monte Carlo dropout, conformal‑style prediction sets, applicability‑domain analysis, and SHAP‑guided toxicophore discovery with occlusion controls. On a leakage‑controlled test fold, a five‑seed soft‑voting ensemble achieved MCC 0.44, AUROC 0.83, and AUPRC 0.58, outperforming four ECFP4‑based shallow baselines across all endpoints.

By Magnus H. Str{\o}mme, Alex G. C. de S\'a, David B. Ascher
benchmarkssafety
More like this →
arXiv Computation and Language
Sep 1

MURANO: Design, Run, and Reproduce Mechanistic Interpretability Experiments as Composable Pipelines

MURANO is an open‑source framework that enables researchers to design, run, and reproduce mechanistic interpretability experiments on large language models. It unifies the five key stages—loading, recording, attribution, intervention, and evaluation—into composable pipeline steps that exchange named artifacts and use canonical addresses for interoperability. The authors demonstrate the framework with reproductions of existing studies and a sparse autoencoder case study, showing its practical applicability across disciplines.

By Alireza Bayat Makou, Emirhan B\"oge, Phu Gia Hoang, Federico Tiblias, Jingcheng Niu, Subhabrata Dutta, Richard Eckart de Castilho, Iryna Gurevych
llmssafety
More like this →
arXiv Computation and Language
Sep 1

CARE: Privacy-Compliant Agentic Reasoning with Evidence Discordance

The paper introduces MIMIC-DOS, a dataset derived from MIMIC-IV that focuses on ICU cases where patient symptoms and medical signs are discordant. It presents CARE, a privacy‑compliant multi‑stage agentic reasoning framework that uses a proprietary LLM to generate structured categories and transitions, while a local LLM performs evidence acquisition and decision‑making. In retrospective evaluations on MIMIC‑DOS, CARE outperforms other LLMs and agentic workflows, demonstrating stronger handling of conflicting clinical evidence while preserving patient privacy.

By Haochen Liu, Weien Li, Rui Song, Zeyu Li, Chun Jason Xue, Xiao-Yang Liu, Sam Nallaperuma-Herzberg, Xue Liu, Ye Yuan
llmsagentssafety
More like this →
arXiv Computer Vision
Sep 1

Coarse to Fine: Iterative Adversarial Neural Cellular Automata for Medical Image Synthesis

The paper introduces StyleGANCA, a lightweight neural cellular automata (NCA) based generative adversarial network designed for medical image synthesis. By combining a StyleGAN-inspired mapping network with adaptive style modulation in a multi-scale NCA framework, the model achieves high-quality image generation with far fewer parameters than existing adversarial, variational, diffusion, and NCA baselines. Experiments on BloodMNIST and PathMNIST show competitive FID and KID scores, and the synthetic images preserve class-specific information, effectively supporting downstream multi-class classifier training.

By Anh Thi Luu, Nick Lemke, Anirban Mukhopadhyay
diffusionbenchmarkssafety
More like this →
arXiv AI
Sep 1

Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations

arXiv:2608.28820v1 Announce Type: new Abstract: Computational pathology (CompPath) is transforming medicine by leveraging artificial intelligence (AI) algorithms to support diagnosis, prognosis, and...

By Shubham Innani, Suhang You, Adam Shephard, Bhakti Baheti, Francesco Ciompi, Joe Yeong, Nasir Rajpoot, Michael Feldman, Solene Florence Kammerer-Jacquet, Dimitrios Makris, Geert Litjens, Anne L. Martel, Jana Lipkova, April Khademi, Spyridon Bakas, for the MICCAI SIG-CompPath
safety
More like this →