arXiv AI

Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation

arXiv AI

Sep 2

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

arXiv:2609.00866v1 Announce Type: cross Abstract: The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pa...

By Yumi Lee, Harim Oh, Hyoryung Kim, Minji Kim, Eunsu Kim, Hyeseong Lee, Junya Fukuoka, Andrey Bychkov, Jijgee Munkhdelger, Rajiv Kumar Kaushal, Ayushi Sahay, Rajni Yadav, Bharathi Prabakaran, Sulen Sarioglu, Serdar Balc{\i}, Ilknur Turkmen, Yuri Tolkach, Christian Harder, Julian Westerdorf, Reinhard Buettner, Audun Ljone Henriksen, Sepp De Raedt, Byung Hyun Lee, Sungjin Lim, Joohoon Lee, Gwanghyun Kim, Se Young Chun, Suryakant Singh, Saarthak Kapse, Prateek Prasanna, Kyung A Kim, Yousun Kang, Sehwan Yoo, Sungman Hong, Shubham Innani, Michael Feldman, Spyridon Bakas, Ujjwal Baid, Prasad Dutande, Suhas Gajare, Bhakti Baheti, Serkan S\"okmen, Ece Tu\u{g}ba Cebeci, Ahmet Hal{\i}c{\i}, Musa Balc{\i}, Kardelen Pe\c{c}enek, Srividhya Sainath, Kyongseok Jang, Messi H. J. Lee, Noorul Wahab, Bodong Du, Jiaming Zhang, Qixiang Zhang, Jang-Hwan Choi, Sangjeong Ahn

llms rag multimodal benchmarks safety

Hugging Face Trending Papers

Sep 1

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the sc...

llms rag multimodal benchmarks safety

arXiv Computer Vision

Aug 25

Expert-level vision-language foundation model for real-world radiology and comprehensive evaluation

arXiv:2409.16183v2 Announce Type: replace Abstract: Radiology is a vital and complex component of modern clinical workflow and covers many tasks. Recently, vision-language (VL) foundation models in m...

By Xiaohong Liu, Guoxing Yang, Yulin Luo, Jiaji Mao, Xiang Zhang, Haibo Wang, Zhiyang He, Ming Gao, Shanghang Zhang, Jun Shen, Guangyu Wang

multimodal benchmarks

arXiv AI

Jul 21

AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis

arXiv:2607. 16260v1 Announce Type: cross Abstract: Multimodal survival analysis utilizing whole slide images (WSIs) and genomic profiles is fundamental for cancer prognosis.

By Jialong Zhong, Tingwei Liu, Baokun Yue, Jingjing Li, Yongri Piao, Miao Zhang, Leiye Liu, Jiahong Jiang, Wei Ji, Huchuan Lu

multimodal

Hugging Face Trending Papers

Jul 27

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine-grained and factualness-driven assessment. In this paper, we introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding that systematically addresses these limitations.

llms rag agents nlp multimodal benchmarks

arXiv AI

Jul 28

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

arXiv:2607. 24743v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine-grained and factualness-driven assessment.

By Hangjie Yuan, Yichen Qian, Zhiwei Tang, Xianzhe Xu, Lirong Wu, Sicheng Yang, Jinwang Wang, Pengju Wang, Zhitao Zeng, Yizeng Han, Yan Xing, Shengxuan Luo, Tao Feng, Qing Xie, Weigen Yao, Yi Yang, Zuozhu Liu, Jiasheng Tang, Shaocheng Wang, Jitao Wang, Jiahong Dong, Weihua Chen, Feng Xu, Fan Wang

llms rag agents nlp multimodal benchmarks

arXiv Machine Learning

Jul 14

LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

arXiv:2607. 11257v1 Announce Type: cross Abstract: Pathology Foundation Models (PFMs) offer powerful Whole Slide Image (WSI) representations but suffer from massive computational costs.

By Gangsu Kim, Won-Ki Jeong

nlp efficiency multimodal safety

arXiv Computer Vision

Sep 2

SlideMix: Enhancing Whole Slide Image Analysis via Multimodal Shuffling

arXiv:2609.00396v1 Announce Type: new Abstract: Histopathological whole slide images (WSIs) are central to cancer diagnosis, but their gigapixel scale, tissue heterogeneity, weak slide-level supervis...

By Chad Wong, Sicheng Chen, Tianyi Zhang, Enhui Chai, Yueming Jin, Zeyu Liu, Fei Xia

llms rag multimodal

arXiv AI

Jul 17

ReportMedSAM: Guiding Segmentation Through Radiology Reports

arXiv:2607. 14116v1 Announce Type: cross Abstract: Free-form radiology reports contain rich clinical descriptions, yet converting them for reliable segmentation remains challenging due to the inherent variability of natural language.

By Anghong Du, Theodoros N. Arvanitis, Colin Watts, Alejandro F. Frangi, Le Zhang

rag computer-vision multimodal

arXiv AI

Aug 18

From Generalist to Specialist: A Context-Fusion Framework for Endoscopic Polyp Reporting with a Frozen VLM

arXiv:2608. 15580v1 Announce Type: new Abstract: Reliable endoscopic polyp reporting requires integrating quantitative lesion sizing, standardized Paris classification, and clinically meaningful morphological description within a single record.

By Ruijie Yang, Yan Zhu, Peiyao Fu, Siyuan Li, Te Luo, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang

llms multimodal

arXiv Computer Vision

Sep 1

Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction

arXiv:2510.06113v2 Announce Type: replace Abstract: Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which red...

By Shuo Jiang, Zhuwen Chen, Liaoman Xu, Yanming Zhu, Changmiao Wang, Jiong Zhang, Feiwei Qin, Yifei Chen, Zhu Zhu

multimodal safety

arXiv Computer Vision

Sep 3

AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $\alpha$-Corrected Binary Cross Entropy and Factorized Latent Supervision

AlphaRAD introduces a grounded zero‑shot classification framework for chest radiology that leverages structured medical concepts extracted from reports and a novel α‑Corrected Binary Cross‑Entropy loss to reduce in‑batch noise. It also presents FLaS, a lightweight cross‑modal fusion module that factorizes VLPM representations into independent subspaces, improving spatial grounding without adding parameters. The method achieves state‑of‑the‑art performance on 16 classification benchmarks and sets new records on several grounding, phrase‑grounding, and segmentation datasets.

By Jianzhong You, Yuan Gao, Chris McIntosh

llms computer-vision multimodal benchmarks safety

Related stories