← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 24

Six misconceptions about large language models: A minimal model and diagnostic taxonomy

The article presents a minimal working model for large language model (LLM) systems, emphasizing four key distinctions—pretraining vs. deployment, distribution vs. samples, types of memory, and task competence vs. agency. Using this framework, it diagnoses six common misconceptions about LLMs (next‑token prediction, regression to the mean, training‑data regurgitation, model memory, alignment, and understanding), explaining what each misconception captures correctly, where it conflates distinctions, and the implications for evaluation, design, and governance. The model is applied to AI policy language, illustrating how policy can misrepresent these distinctions and offering a diagnostic toolkit to correct such errors.

By Zhicheng Lin
llmsagentssafety
More like this →
arXiv AI
Aug 24

Fuzzy-MoE: Interpretable Regime-Conditioned Expert Routing for Non-Stationary Multivariate Time Series Forecasting

The paper introduces Fuzzy-MoE, a fuzzy logic‑based Mixture‑of‑Experts model that jointly identifies latent temporal states and routes forecasts to appropriate experts in non‑stationary multivariate time series. By combining local convolutional dynamics with global segmented statistics, the dual‑view fuzzy router uses learnable Gaussian membership functions to compute expert activation strengths, enabling explicit IF‑THEN rule‑based expert selection. Experiments on several public benchmarks show that Fuzzy‑MoE outperforms mainstream forecasting methods while providing interpretable routing diagnostics.

By Lan Guo, Jie Xiao, Zhao Su, Jun Shen, Haoran Li, Weixia Ma, Qingguo Zhou, Binbin Yong
benchmarkssafety
More like this →
arXiv Machine Learning
Aug 24

Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers

The paper introduces Latent-Posterior Alignment (LPA), a phenomenon where predictive uncertainty in Graph Neural Networks with Bayesian output layers decreases as latent representations align with lower‑variance posterior directions, even without posterior variance contraction. Through interventional experiments, the authors demonstrate LPA’s functional role in shaping uncertainty and propose Alignment‑Guided Learning (AGL) to explicitly promote this alignment during training. AGL reduces predictive uncertainty, preserves accuracy, and improves structural calibration, ensuring model confidence reflects data density.

By Suk Hoon Choi, Damdae Park, Junhyuk Choi, Hyein Jung, Changsoo Kim, Ung Lee, Kyeongsu Kim
safety
More like this →
arXiv Computer Vision
Aug 24

Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planning

This study introduces a two‑stage vision‑language model framework to assess the clinical quality and usability of late gadolinium enhancement (LGE) cardiac MRI images used for atrial fibrillation ablation planning. The first stage employs a fine‑tuned VLM to generate structured radiology‑style reports on five quality criteria—Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under‑segmentation Severity—while the second stage uses a GPT‑based reasoning module to convert these reports into structured quality scores and a binary decision on clinical usability. Evaluated on a curated dataset of 60 image‑slice and text‑pair annotations from 20 patients, the InternVL2 model achieved the highest criterion‑level accuracy, and DeepSeek reached perfect agreement on the clinical usability decision.

By Bipasha Kundu, Abhishek Chaturvedi, Axel W. E. Wismueller, Richard Simon, Cristian A. Linte
llmscomputer-visionfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Aug 24

Explainable Deepfake Detection with Feature-robust Augmentation and Evidence-grounded Explanation Optimization

Explainable Deepfake Detection with Feature-robust Augmentation and Evidence-grounded Explanation Optimization proposes a new framework that improves deepfake detection and interpretability. The approach introduces Feature-robust Augmentation—diversified degradation-aware strategies combined with supervised contrastive learning and a mean-teacher architecture—to maintain accuracy on low-quality images. For explanations, it employs evidence-grounded preference optimization, guiding the model to focus on genuine manipulation traces by learning from chosen-rejected explanation pairs that omit evidence or inject irrelevant details. The method achieved first place in the ACM Multimedia 2026 Explainable Deepfake Detection Challenge and is publicly available on GitHub.

By Zhu Xu, Jiaqi Tang, Pokai Chen, Yuxin Peng, Yang Liu
roboticssafety
More like this →
arXiv Machine Learning
Aug 24

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment

The paper introduces an LLM-as-a-Judge framework for evaluating the outputs of an agentic drug discovery assistant, ChatInvent, deployed at AstraZeneca. It defines four quality dimensions—Completeness, Relevancy, Structural Clarity, and Scope Adherence—alongside deterministic Tool Call Correctness checks, and validates the judge against five expert annotators. After optimizing the best-performing judge with few-shot demonstrations, alignment with human majority votes improves from 0.80 to 0.86, and the framework reveals that informal question phrasing does not degrade output quality.

By Emma Granqvist, Roc\'io Mercado, Samuel Genheden
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Aug 24

Making Deployments Safe at Meta: Health Checks for Continuous Change-Safety

The paper outlines Meta’s deployment time health check infrastructure, called Service Health Checker, which mediates the trade‑off between rapid release velocity and system reliability across thousands of heterogeneous services. It explains how check authors compose templated metric queries, thresholds, and workflow predicates, and how the system integrates with tiered and phased rollouts to trigger automatic rollback on regressions. The authors also discuss operational challenges that arose at scale—such as noise, alert fatigue, drift, and uncovered regressions—and the measurement, tooling, and default improvements they implemented to address these issues, concluding with lessons learned and future directions including AI‑assisted tuning.

By Prakash KL, Anton Korenkov, Uttam Thakore, Christopher Hegre
safety
More like this →
arXiv Computer Vision
Aug 24

ConceptFormer: Learning Adaptive Latent Concepts for Query-Document Alignment in Visual Document Retrieval

arXiv:2608.15698v2 Announce Type: replace Abstract: Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from docume...

By Chunyi Peng, Zhipeng Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Sen Mei, Yubo Sun, Yongheng Zhang, Jie Zhou, Yu Gu, Ge Yu, Maosong Sun
llmsragcomputer-visionmultimodalbenchmarkssafety
More like this →
arXiv Computer Vision
Aug 24

Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation

arXiv:2606.13971v3 Announce Type: replace Abstract: While personalizing Image-to-Video (I2V) diffusion models with specific visual effects is increasingly demanded for high-end generation, current pr...

By Xiaomeng Yang, Yanyu Li, Gordon Guocheng Qian, Ivan Skorokhodov, Viacheslav Ivanov, Avalon Vinella, Xuan Zhang, Yanzhi Wang, Sergey Tulyakov, Anil Kag
diffusionfine-tuningsafety
More like this →