SAP-ViT is a Vision Transformer that replaces the traditional global classification token with direct aggregation of last‑layer patch embeddings, enabling class predictions to be decomposed into patch‑level contributions during the forward pass. The model incorporates sparsity regularization to produce concise, readable evidence and introduces an input‑space back‑mapping mechanism to identify which input patches influence the output. Experiments on diverse medical imaging datasets show that SAP‑ViT retains strong classification performance while delivering consistent, interpretable patch‑level explanations, with ablation studies confirming the benefits of both sparsity and back‑mapping.
arXiv:2609.25978v1 Announce Type: cross Abstract: Saliency maps are widely used to explain deep learning predictions in medical imaging, yet visually plausible explanations do not necessarily reflect...
arXiv:2608. 14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools to decompose internal representations into sparse and more interpretable features.
HierViT is a hierarchical Vision Transformer designed for medical image classification that balances strong predictive performance with interpretability. The model first predicts clinically relevant visual appearance attributes and then uses a hierarchical reasoning process to infer the final diagnosis. Attribute‑specific prototypes and attention maps are visualized to provide complementary views of the model’s reasoning, linking representative image examples and spatial attention to semantically defined visual features.
The paper introduces Expandable Modality Experts, a medical vision‑language framework that uses an expandable bank of modality experts and a Multiway Transformer to combine shared visual computation with deterministic routing and a separate text pathway. It employs a modality‑aware visual tokenizer for masked pretraining on 19.4 million unpaired image rows across seven modalities, followed by contrastive alignment on 3.23 million image‑text pairs. With a ViT‑Large backbone, the method achieves state‑of‑the‑art results in 24 of 25 benchmark settings across nine modalities, including zero‑shot classification, linear probing, and retrieval tasks, while preserving existing representations.
The paper introduces FAD‑ViT, a vision transformer that decouples spatial and feature axes of global attention, making feature‑axis attention the primary operator. Experiments on 23 medical imaging tasks show that shifting global attention from spatial to feature leaves from‑scratch performance unchanged but significantly boosts transfer learning, achieving up to three times the gain with fewer parameters and lower compute. The study demonstrates that prioritizing feature‑axis attention is a simple, cost‑effective design choice for improving medical image transfer.
arXiv:2606. 08156v1 Announce Type: cross Abstract: Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.
arXiv:2608.08374v3 Announce Type: replace Abstract: Transformer models are increasingly used for whole-slide image analysis in computational pathology. Yet, WSIs differ fundamentally from natural ima...
Learning across medical imaging modalities requires making use of images without paired text and accommodating new domains while retaining existing representations. We propose a medical vision-languag...
arXiv:2512. 21414v2 Announce Type: replace-cross Abstract: Recent tool-use frameworks powered by vision-language models (VLMs) improve image understanding by grounding model predictions with specialized tools.
Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images generate prohibitively long visual-token sequences with considerable spatial and inter-slice redundancy.
arXiv:2609.38362v1 Announce Type: new Abstract: Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining dis...
arXiv:2608.21300v1 Announce Type: new Abstract: Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot se...