arXiv Computer Vision

LUX: A Lesion-Aware Graph-Conditioned Visual - Language Architecture for Explainable Endoscopic Captioning

arXiv AI

Aug 6

EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment

arXiv:2608. 04472v1 Announce Type: cross Abstract: The development of foundation models (FMs) is crucial for advancing endoscopic image analysis.

By Zhenyu Yi, Jianwei Xu, Yue Hu, Zhongwei Qiu, Sijing Li, Liang Huang, Bin Lv, Ling Zhang, Yingda Xia

efficiency multimodal safety

arXiv AI

Sep 10

DL$^3$M: A Vision-to-Language Framework for Expert-Level Medical Reasoning through Deep Learning and Large Language Models

arXiv:2512.13742v3 Announce Type: replace-cross Abstract: Medical image classifiers detect gastrointestinal diseases well, but they do not explain their decisions. Large language models can generate...

By Md. Najib Hasan (Wichita State University, USA), Imran Ahmad (Wichita State University, USA), Sourav Basak Shuvo (Khulna University of Engineering and Technology, Bangladesh), Md. Mahadi Hasan Ankon (Khulna University of Engineering and Technology, Bangladesh), Nazmul Siddique (Ulster University, UK), Hui Wang (Queen's University Belfast, UK)

llms computer-vision benchmarks

arXiv Computer Vision

Oct 16

Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology

The paper introduces Locked Image Multiple Instance PreTraining (LIMIT), a method that aligns slide‑level pathology reports with patch‑level visual features by freezing a vision encoder and fine‑tuning a text encoder. LIMIT uses report embeddings as cross‑attention queries over bags of patch features, optimizing slide representations via InfoNCE. Applying LIMIT, the authors create CONCH‑Z, which achieves state‑of‑the‑art zero‑shot performance on 22 pathology tasks, outperforming vision‑language encoders trained on curated caption datasets and narrowing the gap with larger slide‑level foundation models.

By Daniel Shao, Dongmin Bang, Luca L. Weishaupt, Sophia J. Wagner, Nic G. Reitsam, Ming Y. Lu, Long P. Le, Richard J. Chen, Faisal Mahmood

rag fine-tuning multimodal benchmarks

arXiv Computation and Language

Aug 21

Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment

arXiv:2608. 19825v1 Announce Type: cross Abstract: Medical image captioning is a technique that accelerates early-stage diagnostic workflows and enhances the interpretability of medical diagnostic AI systems.

By Yunseo Lee, Hyun Jun Kim, Heeseung Shin, Changwon Lim

llms rag multimodal safety data-engineering

arXiv Computation and Language

4d ago

MIS-VQA: An Annotation-Driven Surgical Vision-Language Dataset for Complex Surgical Understanding

arXiv:2610.34210v1 Announce Type: cross Abstract: Modern surgical AI increasingly requires multimodal understanding rather than isolated perception, yet existing datasets remain largely limited to si...

By Mohamed Hamdy, Iheb Zouari, Fatmaelzahraa Ahmed, Muraam Abdel-Ghani, Muhammed Arsalan, Ponnuthurai Suganthan, Khalid Al-Jalham, Abdulaziz Al-Ali, Shidin Balakrishnan

llms computer-vision nlp efficiency multimodal benchmarks

arXiv AI

Sep 10

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

arXiv:2601.06847v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We po...

By Mengmeng Zhang, Xiaoping Wu, Hao Luo, Fan Wang, Yisheng Lv

llms computer-vision multimodal

arXiv AI

Oct 7

MedCORE: Criteria-Grounded Clinical Reasoning for Interpretable Medical Image Diagnosis

arXiv:2610.08528v1 Announce Type: cross Abstract: Clinical diagnosis is inherently a structured reasoning process, yet existing deep learning models often bypass this structure by mapping image featu...

By Asim Khan, Samee Ullah Khan, Dwarikanath Mahapatra

llms multimodal

arXiv AI

Aug 18

From Generalist to Specialist: A Context-Fusion Framework for Endoscopic Polyp Reporting with a Frozen VLM

arXiv:2608. 15580v1 Announce Type: new Abstract: Reliable endoscopic polyp reporting requires integrating quantitative lesion sizing, standardized Paris classification, and clinically meaningful morphological description within a single record.

By Ruijie Yang, Yan Zhu, Peiyao Fu, Siyuan Li, Te Luo, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang

llms multimodal

arXiv AI

Sep 10

Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation

SCOUT is a concept‑grounded multimodal transformer that generates whole‑slide pathology reports by integrating local histological patterns, whole‑slide context, and expert‑curated diagnostic concepts. It uses evolving visual representations and recursively updated slide‑ and concept‑conditioned representations, with separate attention pathways during decoding that are fused adaptively for each token. Evaluated on TCGA‑BRCA, HistAI, and REG‑2025, SCOUT outperformed existing methods, improving BLEU, METEOR, and ROUGE‑L scores and raising the Clinical Report Quality Score on REG‑2025.

By Suryakant Singh, Saarthak Kapse, Joel Saltz, Prateek Prasanna

llms multimodal

arXiv AI

Sep 2

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

arXiv:2609.00866v1 Announce Type: cross Abstract: The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pa...

By Yumi Lee, Harim Oh, Hyoryung Kim, Minji Kim, Eunsu Kim, Hyeseong Lee, Junya Fukuoka, Andrey Bychkov, Jijgee Munkhdelger, Rajiv Kumar Kaushal, Ayushi Sahay, Rajni Yadav, Bharathi Prabakaran, Sulen Sarioglu, Serdar Balc{\i}, Ilknur Turkmen, Yuri Tolkach, Christian Harder, Julian Westerdorf, Reinhard Buettner, Audun Ljone Henriksen, Sepp De Raedt, Byung Hyun Lee, Sungjin Lim, Joohoon Lee, Gwanghyun Kim, Se Young Chun, Suryakant Singh, Saarthak Kapse, Prateek Prasanna, Kyung A Kim, Yousun Kang, Sehwan Yoo, Sungman Hong, Shubham Innani, Michael Feldman, Spyridon Bakas, Ujjwal Baid, Prasad Dutande, Suhas Gajare, Bhakti Baheti, Serkan S\"okmen, Ece Tu\u{g}ba Cebeci, Ahmet Hal{\i}c{\i}, Musa Balc{\i}, Kardelen Pe\c{c}enek, Srividhya Sainath, Kyongseok Jang, Messi H. J. Lee, Noorul Wahab, Bodong Du, Jiaming Zhang, Qixiang Zhang, Jang-Hwan Choi, Sangjeong Ahn

llms rag multimodal benchmarks safety

arXiv AI

Jun 2

RL-ACRGNet: Reinforcement Learning-Based Chest Radiology Report Generation Network

arXiv:2606. 02035v1 Announce Type: new Abstract: Medical imaging interpretation is a foundational pillar of modern clinical diagnostics, yet the manual generation of radiology reports remains a time-consuming process prone to interpretation inconsistencies.

By Yogesh Kumar Meena, Saurabh Agarwal, K. V. Arya

rag reinforcement-learning benchmarks

Hugging Face Trending Papers

Sep 1

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the sc...

llms rag multimodal benchmarks safety

Related stories