arXiv Computer Vision

An end-to-end-trained vision-language model for native-language prostate pathology report generation

An end-to-end-trained vision-language model generates prostate biopsy reports in native languages, demonstrated in German. The system uses a tokenizer and model trained from scratch and an automated pipeline that splits composite reports into image-text pairs, producing 17,344 pairs from 2,402 cases without manual annotation. Evaluated on clinical attributes, it achieves 96.2% F1 for malignancy detection and 65.2% for Gleason grading, comparable to an FDA-cleared classifier and validated on external cohorts.

arXiv AI
4d ago

Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction

The paper introduces CAST, a concept-guided artifact suppression tuning framework that uses sparse autoencoders to identify and suppress note-specific artifacts in clinical language models. CAST labels latent features with an LLM-assisted pipeline and ICD‑10 constraints, then fine‑tunes the model while providing post‑hoc per‑concept attributions for auditability. In experiments on MIMIC‑IV discharge‑note mortality prediction, CAST outperforms standard fine‑tuned encoders and competes with strong LLM baselines while offering a feature‑level audit trail of clinical concepts and suppressed artifacts.

By Jin Mu, Guanhua Chen
arXiv AI
Aug 11

Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation

arXiv:2605. 30716v2 Announce Type: replace-cross Abstract: Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings.

By Zhiyuan Yang, Jiahao Cheng, Vincent Quoc-Huy Trinh, Mahdi S. Hosseini
arXiv Machine Learning
1d ago

SemEnrich: Self-Supervised Semantic Enrichment of Radiology Reports for Vision-Language Learning

SemEnrich introduces a self‑supervised method to enrich radiology reports by clustering sentences semantically and adding positive or neutral observations from different clusters. The enriched data consistently improves supervised fine‑tuning across multiple vision‑language metrics, with gains ranging from 3% to 7.5% on COMET, Bert, Sentence Bleu, CheXbert‑F1, and RadGraph‑F1. The authors also demonstrate that incorporating cluster information into the reward design for GRPO training yields additional performance boosts.

By Halil Ibrahim Gulluk, Olivier Gevaert
arXiv Machine Learning
Aug 18

Comprehensive language-image pre-training for 3D medical image understanding

arXiv:2510. 15042v3 Announce Type: replace-cross Abstract: In the 3D medical image domain, vision-language pre-training is used to create vision-language encoders (VLEs) that can support radiologists by retrieving patients with similar abnormalities, predicting likelihoods of abnormality, or, with downstream adaptation, generating radiological reports.

By Tassilo Wald, Ibrahim Ethem Hamamci, Yuan Gao, Sam Bond-Taylor, Harshita Sharma, Maximilian Ilse, Cynthia Lo, Olesya Melnichenko, Anton Schwaighofer, Noel C. F. Codella, Maria Teodora Wetscherek, Klaus H. Maier-Hein, Panagiotis Korfiatis, Valentina Salvatelli, Javier Alvarez-Valle, Fernando P\'erez-Garc\'ia
arXiv AI
Aug 18

From Generalist to Specialist: A Context-Fusion Framework for Endoscopic Polyp Reporting with a Frozen VLM

arXiv:2608. 15580v1 Announce Type: new Abstract: Reliable endoscopic polyp reporting requires integrating quantitative lesion sizing, standardized Paris classification, and clinically meaningful morphological description within a single record.

By Ruijie Yang, Yan Zhu, Peiyao Fu, Siyuan Li, Te Luo, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang
arXiv Computer Vision
1d ago

Report Supervision

The paper introduces Report Supervision (R‑Super), a framework that uses radiology reports to supervise tumor segmentation models. By incorporating loss functions that align segmentation outputs with report‑derived tumor counts, sizes, and locations, R‑Super improves detection and segmentation performance. Experiments on kidney and pancreatic tumors show up to a 15% increase in F1‑Score and DSC compared to mask‑only training, outperforming methods like CLIP and multi‑task learning.

By Pedro R. A. S. Bassia, Wenxuan Li, Jakob Wasserthal, Jieneng Chen, Xinze Zhou, Zheren Zhu, Chuntung Zhuanga, Sergio Decherchi, Andrea Cavalli, Kang Wang, Yang Yang, Alan Yuille, Zongwei Zhou