AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 24

Evaluating Large Language Model Performance on International Maritime Dangerous Goods Code Compliance

The paper introduces DGEval, a benchmark of 1,678 questions designed to assess large language models (LLMs) on the International Maritime Dangerous Goods (IMDG) Code Amendment 42‑24. It evaluates 13 models from six providers, finding that while the best model surpasses human practitioners on multiple‑choice tasks, all models perform poorly on safety‑critical areas such as stowage, segregation, and regulatory recall. The study concludes that LLMs can aid compliance tasks—especially structured Dangerous Goods List lookups with web search—but human oversight and authoritative source verification remain essential for safety‑critical deployment.

By Alexander Thomas, Hubert P. H. Shum, Darren Nellis, Manli Zhu, Phatpicha Yochum, William Bartle, Daniel Wrightson
arXiv Computer Vision
Aug 24

Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planning

This study introduces a two‑stage vision‑language model framework to assess the clinical quality and usability of late gadolinium enhancement (LGE) cardiac MRI images used for atrial fibrillation ablation planning. The first stage employs a fine‑tuned VLM to generate structured radiology‑style reports on five quality criteria—Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under‑segmentation Severity—while the second stage uses a GPT‑based reasoning module to convert these reports into structured quality scores and a binary decision on clinical usability. Evaluated on a curated dataset of 60 image‑slice and text‑pair annotations from 20 patients, the InternVL2 model achieved the highest criterion‑level accuracy, and DeepSeek reached perfect agreement on the clinical usability decision.

By Bipasha Kundu, Abhishek Chaturvedi, Axel W. E. Wismueller, Richard Simon, Cristian A. Linte
arXiv AI
Aug 24

Trilingual Topic Modeling of Sri Lankan Parliamentary Debates

The paper presents an end‑to‑end framework for extracting and clustering trilingual Sri Lankan parliamentary debates in Sinhala, Tamil, and English. Using LLM‑based text extraction, multilingual embeddings, and density‑based clustering, the authors recover 30 macro‑topics with a cluster purity of 0.673. The temporal patterns of these topics align with major national events such as the 2019 Easter attacks and the 2022 economic crisis, demonstrating the method’s effectiveness where traditional LDA fails.

By Himath Dhanapala, Haren Daishika, Himandhi Kuruppu, Sithija Seneviratne, Ashini Kavindya, Patalee Narasinghe, Sandeepa Weerasekara, Nisansa de Silva, Sandareka Wickramanayake
arXiv Computer Vision
Aug 24

RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space

RecGen3D is a framework that merges feed‑forward reconstruction and diffusion‑based generation to address the trade‑off between reconstruction fidelity and generative plausibility in sparse‑view 3D modeling. By aligning both models in a shared canonical space and using decoupled cooperative learning, the system stabilizes training and allows the reconstruction module to supply canonical geometric anchors while the diffusion generator refines and completes the structure. Experiments show that RecGen3D outperforms existing methods in producing complete and consistent 3D models from sparse observations.

By Zhisheng Huang, Jiahao Chen, Cheng Lin, Chenyu Hu, Hanzhuo Huang, Zhengming Yu, Mengfei Li, Yuheng Liu, Zekai Gu, Zibo Zhao, Yuan Liu, Xin Li, Wenping Wang
arXiv AI
Aug 24

Continuous-Time Quantum Walks based Graph Neural Network

The paper introduces CTQW-GNN, a graph neural network that uses Continuous‑Time Quantum Walks (CTQW) to address two common GNN problems: low‑pass bias on heterophilic graphs and over‑smoothing with deep layers. By exploiting the unitary nature of the CTQW propagator, the model preserves high‑frequency signals and maintains feature norms across layers. Three aggregation modules—CTQW‑based, CTQW‑attention, and a low‑pass GAT branch—combine to handle both heterophilic and homophilic graph structures, supported by spectral‑gap analysis and a Lieb–Robinson‑type bound for walk‑time selection.

By Yuliang Zhan, Zefeng Gao, Jian Li, Yang Liu, Hao sun
arXiv AI
Aug 24

ReCurveflow: A Flow Matching Framework that Learns Curved Reaction Trajectories to Predict Transition State Geometries

ReCurveflow is a flow‑matching framework that learns to predict transition state geometries by training on continuously curved reference paths derived from full NEB bands, rather than straight linear paths. It introduces an off‑path correction mechanism that generates corrective velocity fields when the model encounters geometries off the training path, improving resistance to exposure bias and TS prediction accuracy. Across multiple data splits and evaluation metrics, ReCurveflow outperforms seven baselines and produces reaction trajectories whose energy profiles closely follow the reference NEB path, aiding NEB optimization and demonstrating effective corrective behavior.

By Seungheun Baek, Mogan Gim, Jaewoo Kang
arXiv AI
Aug 24

Deep Learning Models Also Recall Features

The paper discusses how large language models retrieve facts from their weights, proposing that this phenomenon reflects a broader operation termed feature recall. It argues that a linear projection can be interpreted as retrieving stored information scaled by input activations, and demonstrates that feature recall applies across various architectures, contrasting it with the traditional feature combination paradigm. The authors also explore potential mechanistic identification of feature recall cases and suggest new empirical directions for mechanistic interpretability research.

By Pierre Beckmann
arXiv AI
Aug 24

Peer-Voted LLM-Agent Stress Tests Find Feed-Induced Lexical Convergence but No Reliable Matched-Exposure Advantage for Distributed Sources

The study introduces PV‑SST, a peer‑voted social‑platform testbed, and conducts a preregistered matched‑exposure experiment across four topics, four seeds, four model families, and three larger variants, totaling 448 trials. Results show that feeding agents a ranked list of prior peer posts increases lexical similarity in both the core panel and larger variants, but does not produce a reliable advantage in opinion alignment or survival rates. The only robust finding is lexical convergence driven by the peer‑ranked feed, with no consistent coordination benefit across models or topics.

By Rana Muhammad Usman, Dominic Williamson
arXiv AI
Aug 24

Recognizing Artificial Minds: A Philosophical Defense of AI Cognition

The paper defends the 'Whole Hog Thesis', arguing that sophisticated large language models such as ChatGPT are full linguistic and cognitive agents, possessing understanding, beliefs, desires, knowledge, and intentions. It rejects low‑level computational starting points and instead builds its case from high‑level behavioral observations, using Holistic Network Assumptions to link actions to mental states. The authors systematically rebut common objections—such as hallucinations and planning errors—by showing these resemble human fallibility and by challenging the necessity of traditional conditions like embodiment or semantic grounding.

By Herman Cappelen, Josh Dever
arXiv Computer Vision
Aug 24

Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection

Crane is a CLIP‑based framework for zero‑shot anomaly detection that enhances dense localization by adapting the vision encoder with a correlation‑based attention module and conditioning learnable prompts on global image context. It further fuses anomaly‑relevant patch features into the global representation for more sensitive image‑level detection, and a variant called Crane+ leverages DINOv2 spatial correlations for stronger pixel‑level performance. Across seven industrial benchmarks, Crane raises mean image‑level AP by 4.5% and Crane+ boosts mean pixel‑level AUPRO by 9.0%.

By Alireza Salehi, Mohammadreza Salehi, Reshad Hosseini, Cees G. M. Snoek, Makoto Yamada, Mohammad Sabokrou
arXiv Computation and Language
Aug 24

MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation

MedRAGChecker is a claim-level verification framework designed for biomedical retrieval‑augmented generation (RAG). It decomposes generated answers into atomic claims and assesses each claim’s support by combining evidence‑grounded natural language inference with biomedical knowledge‑graph consistency signals. The aggregated claim decisions provide diagnostics that distinguish retrieval and generation failures, such as faithfulness, under‑evidence, contradiction, and safety‑critical errors, and the system is distilled into compact models for scalable evaluation.

By Yuelyu Ji, Min Gu Kwak, Hang Zhang, Xizhi Wu, Chenyu Li, Yanshan Wang
arXiv Computation and Language
Aug 24

Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers

The paper surveys 61 studies on mental‑health AI and identifies a misalignment in how trust is evaluated across disciplines. It proposes a three‑layer framework—human‑oriented, interaction‑oriented, and AI‑oriented trust—and maps stakeholder perspectives onto these layers. The authors argue that future research should focus on calibrating human trust to actual interaction and AI trustworthiness rather than merely maximizing perceived trust.

By Xin Sun, Yue Su, Yifan Mo, Qingyu Meng, Yuxuan Li, Min Chen, Mengyuan Zhang, Saku Sugawara, Charlotte Gerritsen, Sander L. Koole, Koen Hindriks, Jiahuan Pei