AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Sep 24

AWM-VLA: AlignedWorld Modeling for Efficient and Explainable Vision-Language-Action Policies

AWM‑VLA introduces a unified framework that embeds aligned world modeling directly into a diffusion‑transformer vision‑language‑action policy. By adding learnable future tokens aligned with vision‑language embeddings of future observations, the policy can anticipate long‑term consequences while generating actions. The method extends this with an object‑centric alignment objective and a principled weighting scheme, achieving up to 21% higher success rates on RoboCasa and humanoid tabletop benchmarks and producing object‑centric rationales preferred by human raters in 83% of cases.

By An Lanji, Dawei Liu, Jin Li, Haoran Xu, Mei Chen, Yu Tian
arXiv Computer Vision
Sep 24

DMM-Align: Closed-Loop Optimization for 2D-3D Registration with Dual-Role Diffusion

DMM-Align introduces a closed‑loop framework for 2D‑3D registration that jointly refines correspondences, estimates pose, and learns representations using a shared differentiable geometric state. The method employs two diffusion processes: a geometry‑aware diffusion that improves the soft matching matrix for robust correspondence estimation, and a geometry‑conditioned diffusion teacher that feeds pose‑induced supervision back into feature learning. Experiments on 7‑Scenes and RGB‑D Scenes V2 show that DMM‑Align outperforms strong baselines, particularly in low‑overlap and heavily occluded scenarios, demonstrating the value of closed‑loop geometric feedback.

By Chongjian Wang, Junjie Gao
arXiv Computer Vision
Sep 24

GaussianDS: Depth-supervised Semantic Gaussian Splatting for Scene Understanding

GaussianDS introduces a depth‑supervised framework for 3D Gaussian Splatting that jointly optimizes RGB appearance, depth, and compact semantics from scratch. By arranging multi‑view images into a pose‑aware pseudo‑video and propagating view‑consistent masks via SAM2, the method aligns semantic lifting with geometric cues, using depth supervision and edge‑aware refinement to curb semantic drift and boundary leakage. The approach achieves state‑of‑the‑art performance on LERF and 3D‑OVS benchmarks while preserving high‑fidelity reconstruction and enabling downstream tasks such as 3D object removal.

By Yufei Zhang, Chenlu Zhan, Hongwei Wang
arXiv Computer Vision
Sep 24

Diff-RF: Mutually Reinforced Image Registration and Fusion via Degradation-Aware Learning

Diff‑RF is a diffusion‑based framework that jointly performs image registration and fusion while accounting for degradation in multi‑modal images. It first restores modality‑specific degradations within each image, then uses a cross‑modal diffusion module that couples registration and fusion, refining alignment and enhancing complementary information. Experiments on extended datasets show that this coupled approach yields higher registration accuracy and fusion quality under diverse degraded conditions.

By Xunpeng Yi, Zaixi Du, Qinglong Yan, Yibing Zhang, Han Xu, Jiayi Ma
arXiv Computer Vision
Sep 24

BronchoTop: Bronchoscopy Navigation via RGB-Only Topological Localization

BronchoTop is a real‑time, RGB‑only framework that localises a bronchoscope within the bronchial tree without requiring patient‑specific CT scans or external sensors. It uses four modules—lumen detection and tracking, lumen‑branch label association, probabilistic scope location estimation, and switch verification—to estimate the scope’s position relative to a generic airway model. Evaluation on phantom, simulated, and real data shows state‑of‑the‑art accuracy, improving existing approaches by over 20% on real bronchoscopy sequences and providing the first publicly available framework with code and data for further research.

By Clara Tomasini, Ana Cristina Murillo, Luis Riazuelo
arXiv Computer Vision
Sep 24

Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB

The paper introduces a privacy‑preserving approach for semantic segmentation that fuses high‑resolution depth with ultra‑low‑resolution RGB images. A joint 2D framework uses depth to guide RGB reconstruction and RGB‑D segmentation, while an end‑to‑end 2D‑to‑3D pipeline consolidates 2D features for 3D segmentation. Experiments on ScanNet demonstrate superior 2D and 3D performance compared to other privacy‑preserving methods, strong zero‑shot transfer to SUN RGB‑D and SceneNN, and reduced recoverability of sensitive data, with real‑robot tests showing effective object‑goal navigation.

By Xuying Huang, Swithinraj Moses Daniel, Sicong Pan, Sebastian Houben, Maren Bennewitz
arXiv Computer Vision
Sep 24

ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Images

ASAP is an interactive visualization system that helps users identify and analyze deceptive patterns in AI‑generated images. It uses a CLIP‑adapted image encoder to produce interpretable representations and generates masks that highlight influential pixel regions, enabling influence measurement of key deceptive features. The system integrates these techniques into a dashboard for quantifying authenticity‑indicative patterns across collections of authentic and AI‑generated images, supporting comparative analysis of different generative models such as GANs and diffusion models, and its effectiveness is demonstrated through a user study and benchmark applications.

By Jinbin Huang, Yuki Ueno, Chen Chen, Aditi Mishra, Bum Chul Kwon, Zhicheng Liu, Chris Bryan
arXiv Computer Vision
Sep 24

CRISP: Compositional Relations as Invariant Structural Priors for Domain Generalization

CRISP (Compositional Relational Invariance from Spatial Primitives) is an image‑classification framework that decomposes visual recognition into primitive elements and their relational composition. It represents these compositions with soft unary, binary, and ternary predicates over primitive locations and appearance, enabling differentiable spatial and visual alignment learned end‑to‑end. Evaluated on five DomainBed datasets covering style, provenance, and camera‑trap shifts, CRISP achieves new state‑of‑the‑art performance on both benchmarks.

By Dat Nguyen, Duc-Duy Nguyen
arXiv AI
Sep 24

Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure

Large Reasoning Models (LRMs) use explicit chain‑of‑thought reasoning and large context windows to perform complex tasks, but these features create new attack surfaces. The paper introduces SRCF, an attack that steers LRMs by prepending counter‑aligned few‑shot conversations with explicit CoT traces, causing unsafe outputs on harmful queries and unwarranted refusals on benign ones, without needing model internals. To counter this, the authors propose ARCF, a post‑training defense that exposes models to counter‑aligned conversational contexts while enforcing aligned targets, improving safety and helpfulness without harming utility.

By Xiangyu Zhou, Saleh Zare Zade, Dongxiao Zhu
arXiv AI
Sep 24

Ask Which, Not How Good: Sizing Benchmarks Scored by an LLM

The study analyzes 373,019 judgments from LLM‑scored benchmarks, decomposing variance into system, item, judge, and interaction components via generalizability theory. It finds that with a single judge, generalizability converges to a ceiling determined by the system‑by‑judge variance, which is substantially lower in pairwise preference settings, allowing one judge to suffice. The research also reveals significant biases in presentation order and highlights that many published win‑rate claims fall below the measured floor of the benchmarks.

By Atul Anand
arXiv AI
Sep 24

Hard Negatives Reveal What Easy Negatives Hide: Cross-Lingual Harmfulness Representations Degrade with Resource Tier Under Hard Negatives

The study investigates how safety alignment in large language models, trained mainly in English, transfers to other languages. While models show near-perfect harmfulness detection (AUROC > 0.98) using unrelated harmless prompts (easy negatives), performance drops sharply in low‑resource languages when using surface‑similar benign prompts (hard negatives). This degradation persists across multiple languages and models, indicating that easy‑negative evaluation alone cannot confirm cross‑lingual harmfulness representation quality.

By Paras Balani, Subhrakanta Panda
arXiv AI
Sep 24

Beyond Poetry: Can Large Language Models Generate Classical Arabic Maqamat?

arXiv:2609. 28245v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong performance in creative text generation, yet their ability to produce culturally grounded and stylistically constrained literary forms remains underexplored.

By AbdulRahman A. Morsy (Department of Computer Science, School of Engineering and Applied Sciences, George Washington University, Washington DC, United States), Aya Zirikly (Department of Computer Science, School of Engineering and Applied Sciences, George Washington University, Washington DC, United States, Center for Speech and Language Processing, Whiting School of Engineering, Johns Hopkins University, Baltimore MD, United States)
arXiv Machine Learning
Sep 24

A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems

The paper introduces a systematic benchmark for evaluating explainable methods that attribute temporal interactions in sequential recommendation systems. Using a dual-model masking metric, it assesses ten XAI techniques across CNN, Transformer, SASRec, and BERT4Rec backbones on KuaiRand and MovieLens datasets, revealing that gradient-based methods like GradientSHAP and Integrated Gradients are the most faithful and robust. It also finds that raw attention weights are unreliable, while gradient-weighted attention works better on short sequences but degrades on longer horizons, and that faithful methods capture genuine task structure rather than recency or popularity bias.

By Akash Pandey, Kanisha Shah, Addrish Roy, Dwipam Katariya, Hongyangyang Shi, Amanda Ding, Kalanand Mishra, Pranab Mohanty
arXiv Machine Learning
Sep 24

Binary Quantized Neural Network Training Is W[1]-Hard Parameterized by Input and Output Dimensions

The paper proves that training a binary quantized neural network (2-QNNT) is W[1]-hard when parameterized solely by the sum of input and output dimensions, α+ω. This hardness result holds even for zero training error on a specially constructed dataset where each input equals its target and the examples form a coordinate‑wise prefix chain. The proof reduces from DAG edge‑disjoint paths, employing a one‑flip routing equivalence that links activation transitions to vertex‑disjoint paths in the network.

By Tao Jiang, Minbo Gao, Shaowei Cai
arXiv Machine Learning
Sep 24

A Discrepancy-Based Perspective on Dataset Condensation

The paper introduces a unified framework for dataset condensation (DC) that generalizes existing methods by using discrepancy measures to quantify the distance between probability distributions. It extends the traditional goal of DC—creating a small synthetic dataset that preserves generalization—to include additional objectives such as robustness and privacy. The framework positions DC as a formal approximation problem, broadening its applicability across different machine learning regimes.

By Tong Chen, Raghavendra Selvan
arXiv Computation and Language
Sep 24

When Helpful Context Leaks: Privacy Risks in Domain-Adapted ASR

SpeechLLMs used in professional settings often undergo domain customisation through prompts or fine‑tuning, which can inadvertently cause the model to transcribe phonetically similar words from its context or training data, leaking private information. The authors systematically investigate this overlooked privacy risk, creating benchmarks to measure leakage rates for both prompting and fine‑tuning, and find that both mechanisms cause measurable leakage that compounds when combined. They evaluate a prompt‑level mitigation strategy and analyse the accuracy‑leakage trade‑off, concluding that fine‑tuning without context prompts offers the best balance between performance and privacy.

By Maike Z\"ufle, Jan Niehues
arXiv Computation and Language
Sep 24

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

The paper introduces a trilingual spoken hallucination detection benchmark covering English, Russian, and Kazakh news, with 12,013 samples that include synthetic alterations and severity levels, as well as 290 fact‑checked misinformation items. Detectors are evaluated in a reference‑free setting on text, ASR transcripts, and audio, revealing that most models underperform baseline classifiers, except Gemma‑3n on transcripts. Synthetic‑trained detectors achieve high macro‑F1 scores on real‑world misinformation, but Russian provenance analysis highlights model‑dependent signals that confound synthetic benchmarks.

By Meruyert Aristombayeva, Jason S. Lucas, Chaewan Chun, Dongwon Lee
arXiv Computer Vision
Sep 24

Gender Bias in Vision-Language In-Context Learning

The paper investigates how in‑context learning (ICL) in large vision‑language models (LVLMs) can amplify gender bias. Using the VL‑BICLE framework, the authors show that gendered ICL demonstrations shift model bias toward the demonstrated gender, especially in tasks involving gendered language such as image captioning and pronoun prediction. They find that similarity‑based retrieval does not mitigate this bias and that replacing real images with synthetic ones from stable diffusion reduces bias without hurting caption quality.

By Tong Xiang, Noa Garcia, Yuta Nakashima
arXiv AI
Sep 24

Agentic Governance and Adversarial Verification for Policy-Constrained LLM Healthcare Appeal Generation

The paper introduces AGVF, a multi‑agent framework for generating medical‑necessity appeals that must adhere to payer policy and evidence constraints. AGVF treats appeal synthesis as a Constrained Markov Decision Process involving five agents—policy formalization, evidence retrieval, gap analysis, adversarial critique, and gated synthesis—and proves that refining the policy constraint graph monotonically reduces evidence deficiency. A deterministic citation‑grounding gate ensures no unsupported assertions enter the shared state, and validation on 1,000 synthetic cases shows zero citation violations and monotonic deficiency reduction.

By Harshil Lodhiya, Alex McManus, Reese Walker
arXiv AI
Sep 24

Discovery of fully efficient fault indicators along a data-based diagnosis process

The paper presents DT4X+, an improved diagnosis algorithm that builds on DT4X by refining training set construction and symbolic‑regression loss to better separate target classes while maintaining coherence among non‑target classes. This results in relations that fully align with analytical redundancy relation properties, yielding more informative decision‑tree splits, enhanced robustness, and superior performance on dynamic‑system datasets. Experiments on benchmark systems confirm these advantages.

By Igor Bezmaternykh (INSA Toulouse), Louise Trav\'e-Massuy\`es (LAAS-DISCO, Comue de Toulouse, ANITI), Elodie Chanthery (LAAS)