AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
4d ago

Exploring Bad Minima in Transfer Learning

The paper investigates whether strong generalization on a source task guarantees transferability to a related target task in transfer learning. The authors hypothesize a negative answer and conduct adversarial attacks on the transfer learning process to find counterexamples. Their empirical results reveal a strong relationship between source-task generalization and transferability, though they do not rule out the possibility that good source generalization is sufficient for successful transfer.

By Mina Remeli
arXiv Computer Vision
4d ago

TACO: Trajectory-Aware Curvature Observation for Memorization Localization and Mitigation in Diffusion Models

TACO (Trajectory-Aware Curvature Observation) is a framework designed to detect and mitigate memorized content in text-to-image diffusion models. It aggregates curvature responses along the late denoising trajectory and uses latent-guided spatial refinement to produce a memorization map that both localizes and guides targeted mitigation. Experiments show TACO outperforms existing localization methods and improves the balance between memorization suppression, semantic alignment, and perceptual quality.

By Ruxi Gu, Zilei Wang, Wei Wang
arXiv Computer Vision
4d ago

TADP: Dual-Branch Temporally Adaptive Differential Privacy for Expression-Aware Video Anonymization

TADP is a temporally adaptive differential privacy framework designed for privacy-preserving video generation with fixed-length clip-replacement adjacency. It employs a dual-branch architecture to separately extract identity and expression representations, applying temporally adaptive Gaussian mechanisms to each branch and using adaptive Rényi Differential Privacy composition to ensure joint release privacy. The framework converts clip-level RDP guarantees to (ε,δ)-differential privacy, applies DP post-processing, and demonstrates reduced identity leakage, good perceptual quality, temporal stability, and high expression recognition utility on facial expression datasets.

By Feng Xu, Xun Li, Lars Petersson, Yulei Sui, David Ahmedt-Aristizabal, Dadong Wang
arXiv Computer Vision
4d ago

Any Forcing: Any-Step Generality and One-Step Stability for Autoregressive Video Generation

The paper introduces Any Forcing, a framework that enables flexible sampling steps and stable one-step generation for autoregressive video models. It employs a unified adversarial discriminator to align real distribution, inter-step consistency, and first-frame quality, and a training-free adaptive memory selection that preserves temporal relationships. Experiments show high-quality, stable one-step generation at 20.9 FPS for short videos and 24.7 FPS for long videos on a single H100 GPU.

By Jiahua Xu, Yijie Fang, Nannan Wang, Xunhao Lin, Li Cao, Lei Hao, Mingrui Zhu, Fei Gao, Xinbo Gao
arXiv Computer Vision
4d ago

PBSR: Pseudo-Burst Super-Resolution via Zero-Shot Novel View Synthesis

PBSR: Pseudo-Burst Super-Resolution via Zero-Shot Novel View Synthesis proposes a method to enhance single-image super-resolution by generating synthetic burst frames through zero-shot novel view synthesis. The approach uses a viewpoint sampling strategy aligned with real burst data to create realistic parallax, then processes these frames with a BSR backbone that includes geometry-aware alignment and artifact-aware adaptation. Experiments on the BurstSR dataset demonstrate consistent improvements in PSNR, SSIM, and LPIPS compared to a single-frame baseline using the same reconstruction backbone.

By Yuki Kondo, Norimichi Ukita
arXiv Computer Vision
4d ago

With foundation-model features, aggregator complexity does not improve cross-hospital discrimination in gastric whole-slide image classification

The study evaluated whole‑slide image classifiers for gastric cancer across five Korean hospitals, using foundation‑model features and leave‑one‑hospital‑out validation. A hierarchical patch‑region‑slide graph aggregator (CHIME‑MIL) with site alignment and two‑scale fusion achieved a mean AUC of 0.837, outperforming six conventional attention and transformer aggregators that lacked these steps. However, when matched with the same alignment and fusion, CHIME‑MIL did not surpass simple attention‑based pooling, and global max pooling performed best (AUC 0.855). The alignment step, which subtracts each hospital’s mean feature vector without labels, enabled onboarding of a new hospital from only ten unlabeled slides but yielded only a modest 0.002 AUC improvement, and the model’s accuracy gains were inconsistent. "whyItMatters":"The findings show that adding complex aggregation methods does not necessarily improve cross‑hospital discrimination, highlighting the importance of simpler, label‑free onboarding and calibration checks for deployable pathology systems."

By Nheng Vanchhay, Ali Hussain, Khan Ziaullah, Moon-Il Joo, Md Ariful Islam Mozumder, Rashedul Islam Sumon, Won-Ae Lee, Min-Sun Cho, Hyeong-Chan Shin, Yujun Park, Baek Hwan Cho, Nam-Hoon Cho, Heejung An, Heung-Kook Choi, Haeyoun Kang, Hyunki Kim, Hee Cheol Kim
arXiv Computer Vision
4d ago

Defining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks

The paper introduces page‑centric Visual Document Retrieval (VDR), treating pages as the primary indexing units while leveraging document context for relevance assessment. It distinguishes page and document retrieval granularities and three retrieval paradigms—independent‑item, evidence‑set, and iterative—then evaluates 42 existing benchmarks across eight criteria. The analysis reveals that most current VDR research focuses on page‑level independent‑item retrieval, with limited support for document‑level, evidence‑set, and iterative settings, and that realistic user queries and graded relevance judgments are rarely used, highlighting gaps between application needs and evaluation protocols.

By Chongzhi Zhang, Yuqi Zuo, Aixin Sun
arXiv Computer Vision
4d ago

AdaGS: Confidence-Gated Diffusion Adaptation for Unposed Sparse-View 3D Gaussian Reconstruction

AdaGS introduces a confidence-gated diffusion adaptation framework to improve unposed sparse-view 3D Gaussian reconstruction. The method uses an artifact-aware adapter trained on a Gaussian artifact dataset to suppress floating artifacts before diffusion restoration, and employs bidirectional confidence fusion to accept generated regions only when supported by cross-view geometry. Additionally, a Gaussian stabilization strategy reduces weakly inconsistent Gaussians that could cause hallucinations, leading to better rendering fidelity and pose accuracy on three outdoor datasets.

By Beizhen Zhao, Sicheng Yu, Guanzhi Ding, Yu Hu, Hao Wang
arXiv Computer Vision
4d ago

MuCALD-SplitFed: Causal-Latent Diffusion for Privacy-Preserving Multi-Task Split-Federated Medical Image Segmentation

MuCALD‑SplitFed is a multi‑task split‑federated learning framework that combines causal representation learning with latent diffusion to enable privacy‑preserving medical image segmentation across decentralized institutions. It addresses the instability and privacy issues of existing multi‑task federated approaches by reducing information leakage at split points and improving convergence. Experiments demonstrate that MuCALD‑SplitFed outperforms baseline split‑federated, personalized federated, and multi‑task federated methods on segmentation tasks.

By Chamani Shiranthika, Hadi Hadizadeh, Parvaneh Saeedi
arXiv Computer Vision
4d ago

SUREON: A Benchmark and Vision-Language Model for Surgical Reasoning

SUREON is a new video question‑answering dataset and benchmark that tests visually grounded surgical reasoning. It contains 76K training pairs from narrated surgical videos and a 340‑question test set verified by expert surgeons, designed to prevent models from relying on language shortcuts. Baseline models trained on SUREON outperform existing open‑source models and show that surgical visual reasoning remains largely unsolved, with no model exceeding 53% accuracy.

By Alejandra Perez, Anita Rau, Busisiwe Mlambo, Lee White, Aina Brunet, Juan Verde, Juan Caicedo, Jo\"el L. Lavanchy, Adam Schmidt, Chinedu Nwoye, Omid Mohareri, Muhammad Abdullah Jamal
arXiv Computer Vision
4d ago

Knowledge-structured and evidence-aware semantic self-distillation for glomerular pathology foundation model

The paper introduces GloSAGE, a semantic self‑distillation framework that leverages lesion profiles from a pretrained visual teacher to generate structured pathology descriptions for glomerular images. By selecting reliable image‑text anchors and synthesizing knowledge‑structured text, GloSAGE trains a frozen visual encoder with an intermediate semantic transfer layer, enabling zero‑shot lesion recognition, image‑text retrieval, and improved downstream tasks such as disease classification and renal function estimation. Experiments on the GloMillion dataset and seven cohorts demonstrate that GloSAGE outperforms its source visual model in most lesion‑recognition tasks and enhances grading accuracy while maintaining competitive performance on case‑level analyses.

By Qiming He, Jing Li, Zihan Wang, Tian Guan, Yifei Ma, Zimo Zhao, Yanxia Wang, Hongjing Chen, Yexing Zhang, Yizhi Wang, Xinrui Chen, Xitong Ling, Minxi Ouyang, Yujie Xie, Lianghui Zhu, Yiqing Liu, Qingxia Hou, Shuyan Zhao, Xiaoqin Wang, Lili Ma, Qiang Huang, Zhiyuan Shen, Junru Cheng, Siqi Zeng, Chao He, Yonghong He, Zhe Wang
arXiv Machine Learning
4d ago

Lattice-Based Pruning of Recurrent Neural Networks via Poset Modeling: A Corrected Theory and a Matched-Sparsity Evaluation

The paper revisits magnitude pruning for recurrent neural networks, arguing that it neglects the flow of information between neurons. It proposes modeling RNN dependencies as a partially ordered set (poset) and pruning by retaining the meet‑irreducible elements of the resulting dependency lattice, a method that requires a well‑defined dependency order obtained by condensing thresholded graphs into strongly connected components. Experiments on 128‑unit LSTMs trained on sequential MNIST show that the earlier critical‑set criterion fails to preserve any recurrent weights, that the new order‑based score performs no better than simple metrics like degree or PageRank, and that enforcing acyclicity from initialization incurs significant accuracy loss at high sparsity levels. whyItMatters":"The study demonstrates that without imposing a proper dependency structure, pruning recurrent networks can inadvertently remove all recurrent connections, highlighting the need for careful theoretical foundations in sparsity‑aware training."

By Rakesh Sengupta
arXiv Machine Learning
4d ago

Hyperbolic Inverse Reinforcement Learning

Hyperbolic Inverse Reinforcement Learning (Hyperbolic IRL) introduces a reward function that depends on the Hyperbolic distance between feature vectors and a learned parameter vector, offering an alternative to traditional linear or neural network approaches. The method matches state‑of‑the‑art performance on standard benchmarks, performs especially well with limited data and in dense environments, and uncovers complex, non‑separable reward structures in human navigation tasks. Hyperbolic IRL thus enhances both predictive accuracy and interpretability of inferred reward functions.

By Iulia Rusu, Anoop Praturu, Sergei Gepshtein, Tatyana Sharpee
arXiv Machine Learning
4d ago

Variational decomposition autoencoding improves disentanglement of latent representations

The paper introduces variational decomposition autoencoding (VDA), a framework that biases signal decomposition into latent representations. VDA is realized as variational decomposition autoencoders (DecVAEs), encoder-only networks that learn multiple latent subspaces aligned with time‑frequency characteristics. Experiments on simulated data and three speech datasets show that DecVAEs outperform traditional VAEs in disentanglement quality and robustness, producing factor‑specific latent subspaces when generative factors have distinct spectral signatures.

By Ioannis N. Ziogas, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis
arXiv Machine Learning
4d ago

From Training Exposure to Membership Evidence: A Controlled Study of OLMo 2

The study re‑runs the OLMo 2 1B language model with controlled training exposure, swapping selected training chunks with matched documents to assess membership inference. Using a counterfactual run from the same seed, the authors find that a single exposure modestly increases a document’s likelihood but rarely allows reliable identification; the reference‑ratio method achieves an AUC of 0.767 and detects 3.8 % of members at a 1 % false‑positive rate. Even with matched references, document‑level variability limits individual inference, and the authors plan to release checkpoints, log‑probabilities, and evaluation data.

By Ali Banaei, Muhammad Shahab Shahzad, Gerald Q. Maguire Jr., Marco Chiesa, Dejan Kosti\'c
arXiv Machine Learning
4d ago

Learning Robust Social Strategies with Large Language Models

The paper investigates how large language models (LLMs) can be trained to cooperate in multi‑agent social dilemmas. It shows that standard reinforcement learning often leads LLM agents to defect and exploit others, even when they have cooperative priors. By adapting the Advantage Alignment algorithm and introducing a group‑relative baseline, the authors fine‑tune LLMs to achieve higher collective payoffs and resist exploitation, and they present a new Trust‑and‑Split environment that requires natural language communication for optimal outcomes.

By Dereck Piche, Mohammed Muqeeth, Milad Aghajohari, Juan Duque, Michael Noukhovitch, Aaron Courville
arXiv Machine Learning
4d ago

Don't Just Impute, Detect: Task-Aware Diffusion Pipelines for Edge Anomaly Detection

The paper introduces Task‑Aware Diffusion Refiner (TADR), a conditional diffusion model that refines imputed data for edge anomaly detection in IIoT pipelines, addressing the bias of standard MSE‑trained imputers. TADR is trained against a frozen downstream detector’s loss landscape, requiring no extra back‑propagation through sampling, and achieves a 0.338 F1 improvement over the unrefined detector on the SMD dataset. The method’s effectiveness depends on the presence of continuous, redundant sensor channels and is less beneficial for telemetry data dominated by one‑hot command signals.

By Jeongsu Park, Taehyeon Yun, Geonwoo Kim, Yuji Lim, Moohong Min
arXiv Machine Learning
4d ago

Fewer Heads, Fewer Memories: Pruning Redundant Attention Heads Suppresses Verbatim Memorisation

Large language models can memorize and reproduce verbatim training data, raising privacy and IP concerns. Recent theory links attention heads to this memorisation, while empirical work shows many heads are redundant and can be pruned with little performance loss. This study shows that zeroing out a large fraction of redundant heads before inference dramatically reduces memorisation (91.5% drop) and even improves language‑modeling performance on GPT‑2‑XL.

By Aryan Chowdhury, Riza Batista-Navarro
arXiv Machine Learning
4d ago

From the Zoo to the BULLPEN: Sample-Efficient LLM Profiling via Multi-Channel Model Embeddings

The paper introduces BULLPEN, a framework that builds compact embeddings of large language models (LLMs) by jointly analyzing model answers, correctness, and question text. BULLPEN expands on prior work by adding 20 new benchmarks focused on behavior, style, and provenance, and demonstrates that embeddings derived from only 400 questions retain most of the predictive power of full‑pool methods. Across 260 open‑weight models, BULLPEN’s answer‑text encoder achieves higher accuracy on new tasks and preserves over 91% of full‑pool skill with minimal inference cost.

By Russell Ro, Qiming Cui, Reza Abbasi-Asl
arXiv Machine Learning
4d ago

Enhancing Medical Lesion Segmentation via Several Simple Modules

The paper presents three lightweight modules to improve medical lesion segmentation, especially in data‑scarce and computationally constrained settings. The modules include a DT‑Morph Combined Loss for better topological and boundary optimization, a 3D Group Prompt that captures domain‑universal knowledge through volumetric image‑mask and text prompts, and a Many‑to‑One Conditional Segmentation Head that supports fast incremental learning with minimal computational overhead. Ablation studies on abdominal and extra‑abdominal datasets confirm the effectiveness of these components for both organ and lesion segmentation tasks.

By Ruichen Li, Zhao Yao