AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Computer Vision
Aug 25

EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment

EXPL-FR is a lightweight adapter that aligns a vision‑language model’s image encoder with a frozen face‑recognition (FR) embedding space, enabling the FR model to be explained using semantic attribute prompts without any text training. By mapping 978 attribute prompts across 22 categories into the FR space, the method identifies the most detectable concepts—forming a readable semantic signature that better separates identities than the full vocabulary. The approach is evaluated on four FR backbones and two VLM encoders, providing identity‑level, per‑image, and differential explanations, and demonstrates that prompt‑driven audits can rank FR models by per‑ethnicity error and attribute‑change verification cost without requiring labeled data.

By Guray Ozgur, Mustafa Efe Tamyapar, Naser Damer, Fadi Boutros
arXiv Computer Vision
Aug 25

Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition

The paper introduces LaST, a large-small collaborative framework for zero-shot surgical phase recognition. It combines a foundation model that generates frame-level phase priors with a lightweight model that refines predictions through iterative temporal refinement, dynamic quality control, and dual-model cross-learning. Experiments show LaST outperforms baseline and state-of-the-art methods, achieving significant accuracy gains on unseen clinical domains.

By Yiyi Zhang, Ying Zheng, Wenxin Fan, Yu Zhu, Yuchen Yuan, Litao Zhao, Zheng Li, Pheng-Ann Heng
arXiv Computer Vision
Aug 25

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

VISTA is a gradient‑based test‑time alignment framework designed for next‑scale visual autoregressive (VAR) image generation. It optimizes intermediate representations within the frozen transformer to enforce compositional constraints, without altering model weights or requiring extra training. Experiments on two benchmarks and two model scales show that VISTA improves compositional accuracy by up to 20% on a 2B backbone and 6% on an 8B backbone, while preserving image quality and enabling a smaller model to outperform a larger one.

By Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah
arXiv AI
Aug 25

CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulation

arXiv:2604.09746v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are increasingly deployed as autonomous agents, understanding how strategic behavior emerges in multi-agent e...

By Aarush Sinha, Arion Das, Soumyadeep Nag, Charan Karnati, Shravani Nag, Chandra Vadhan Raj, Aman Chadha, Vinija Jain, Suranjana Trivedy, Amitava Das
arXiv AI
Aug 25

TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts

arXiv:2608.22788v1 Announce Type: new Abstract: Large-scale rollouts have become a core component of modern LLM systems, spanning reinforcement learning (RL) post-training, on-policy distillation (OP...

By Tianqi Xu, Lu Lv, Haoyang Huang, Wenjie Huang, Zhanming Shen, Yuhao Shen, Baolin Zhang, Xinyi Hu, Shuang Ge, Jun Dai, Tianyu Liu, Suorong Yang, Zhikai Li, Ye Bai, Jun Zhang, Lei Chen, Yue Li, Mingchen Wan
arXiv AI
Aug 25

Diagnosing Capability Preservation and Task Sensitivity in Memory Augmented Document Classifiers

The paper introduces Protected QL Memory, a dual‑path document classifier that separates a causal local pathway from an associative matrix writer and a finalized memory reader. It evaluates three empirical properties—capability preservation, diagnostic access, and task performance sensitivity—across nine dataset‑seed conditions, finding that writer capability is fully preserved while diagnostic accuracy gaps are large but macro‑F1 scores change minimally. The study concludes that preserving associative capability and diagnostic access does not guarantee downstream reliance on memory, urging separate evaluation of these objectives.

By Isaac Kofi Nti
arXiv Statistics ML
Aug 25

A Commutator Framework for Selective Spectral Alignment in Deep Neural Networks

The paper introduces a finite‑width geometric framework that explains how learned feature geometries are organized, transported, and selectively aligned in deep neural networks. It quantifies incompatibilities among weight‑generated covariance, gates, and backward sensitivities using three families of commutators, and provides exact layerwise identities that decompose these commutators into sources such as downstream transport, adjacent‑layer imbalance, and nonlinear gate‑covariance interactions. The study demonstrates that spectral alignment is a layer‑ and scale‑dependent compatibility phenomenon governed by transport, interaction, cancellation, and possible damping, rather than a universal consequence of training.

By Kaj Nystr\"om
arXiv Computation and Language
Aug 25

Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation

arXiv:2605.11533v4 Announce Type: replace Abstract: Routine clinical check-up reports combine laboratory measurements, physiological assessments, imaging findings and visually structured information,...

By Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei
arXiv Computer Vision
Aug 25

SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space

SketchFlow is a new generative framework for creating high‑quality vector sketches from text prompts. It uses a Gaussian Mixture Model prior in the CLIP latent space and an Optimal Transport Conditional Flow Matching model to map this prior to sketch features, which are then decoded by a Hybrid Diffusion Decoder combining 1D U‑Net and Transformer architectures. The approach achieves superior visual quality and human‑like drawing styles, and supports zero‑shot synthesis for unseen concepts and smooth semantic interpolation.

By Jin Zhou, Hongliang Yang, Pengfei Xu, Hui Huang
arXiv Computation and Language
Aug 25

Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation

The paper investigates how annotator-style rebuttals can manipulate large language model (LLM) moderation systems, either by whitewashing hateful content as normal or smearing normal content as hateful. Using a rejudge protocol that adds decision‑boundary perturbations and adversarial rationales, the authors show that such rebuttals significantly degrade moderation performance, especially in multi‑turn settings. The study finds consistent, model‑specific asymmetries between the two manipulation directions and demonstrates that explicit reasoning prompts and defensive instructions mitigate but do not eliminate the vulnerability.

By Junyu Lu, Kaiyuan Liu, Jingyi Kang, Deyi Ji, Hailong Zhang, Lanyun Zhu, Qi Zhu, Bo Xu, Liang Yang, Hongfei Lin
arXiv AI
Aug 25

Cross-lingual Biography Enrichment via Claim Extraction and Alignment

The paper introduces CLAW-4L, a benchmark of 300 pairs of English and non‑English Wikipedia biographies (French, Chinese, Azerbaijani) focused on women from non‑English contexts, complete with claim annotations and a fine‑grained claim‑pair relation corpus. It proposes a claim‑based enrichment framework that extracts claims from both biographies, aligns them to identify enrichment evidence from the non‑English version, and rewrites the English biography accordingly. Experiments demonstrate that non‑English Wikipedia biographies can improve English biography coverage, though lower‑resource settings still pose challenges.

By Yifei Song, Ziyang Chen, Emil Sayilov, Claire Gardent