arXiv Computer Vision

Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining

arXiv Machine Learning
Oct 12

The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging

The paper introduces TAP-JEPA, a joint‑embedding predictive architecture trained on over 113,000 CT volumes, which achieves performance comparable to leading encoders in scan‑level classification. It demonstrates a predictability bias: while anatomy is largely predictable from context, pathology is not, leading to significant degradation in pathology detection when predictions are applied. The study audits the model’s predictive pathway, showing that pathology representations shift toward control regions and that removing pathology tokens minimally affects downstream decisions, underscoring that contextual predictability alone is insufficient for clinical relevance.

By Fabian Drexel, Bernhard Kainz, Jakob N. Kather, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem, Cosmin I. Bercea
Hugging Face Trending Papers
Jun 24

Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography

Vision-language pre-training (VLP) holds great promise for general-purpose medical AI by leveraging radiology reports as rich textual supervision, yet existing methods struggle with 3D CT imaging due to inefficient visual backbones and coarse semantic alignment. To address these issues, we propose a tailored VLP framework featuring three key components: (1) a CNN-ViT hybrid encoder that replaces ViT's patch embedding with a 3D CNN backbone to efficiently capture local anatomical details while preserving global attention and compatibility with pre-trained cross-modal priors; (2) a disease-level contrastive learning mechanism using learnable query tokens to dynamically extract disease-specific semantics from full reports and align them with corresponding visual features, thereby disentangling distinct diseases within the same anatomical region; and (3) a diagnosis-aware prompt strategy that employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap and enhance zero-shot diagnostic reliability.

arXiv Computer Vision
Aug 28

DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT

DALE-CT introduces depth‑aware 2D slice encoders that learn an anatomical world model of chest CT scans without 3D or positional supervision. By sampling self‑supervised views across a physical $z$‑axis slab, the encoder captures how anatomy changes between neighboring slices, enabling it to recover slice ordering and distinguish slices by anatomy alone. The model, trained on a large 287k‑scan corpus, achieves state‑of‑the‑art performance on CT‑RATE and is released with full code and evaluation tools.

By Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty, Caroline N. Leach, Emily B. Collier, V. K. Cody Bumgardner
arXiv AI
Aug 5

CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement

arXiv:2608. 03890v1 Announce Type: cross Abstract: A clinically useful chest X-ray system must go beyond fluent report generation: it should classify findings with tunable decision thresholds, localize them spatially, and derive the anatomical measurements upon which many diagnoses depend.

By Mercy Prasanna Ranjit, Anirban Porya, Sathvik Joel, Niharika Vadlamudi, Nikhilesh Chowdary Eathamukkala, Prasanth V V, Abhyuday Kumara Swamy, Pranay Narhari Umredkar, Pradeep Narayan, Vivek Rajagopal, Tanuja Ganu
arXiv AI
Oct 27

Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference

Meta-CLIP is a metadata‑conditioned contrastive learning framework that incorporates DICOM acquisition information to reweight cross‑modal negatives during medical vision‑language pretraining. By defining a bounded pairwise margin over numerical and categorical DICOM fields, the method emphasizes distinctions among studies with similar imaging conditions while keeping the kernel fixed during optimization. Evaluations on chest radiography and volumetric CT show that Meta‑CLIP improves zero‑shot classification performance, achieving higher macro‑AUROC scores than baseline CLIP models on multiple benchmarks.

By Haozhe Luo, Nirosh Sivanesan, Shelley Zixin Shu, Francesco Nitti, Mauricio Reyes