Hugging Face Trending Papers

Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?

As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnosis ability over given medical images and texts, implicitly assuming that standardized medical images, texts or question-answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources.

arXiv Computer Vision

Sep 7

Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?

The paper introduces the Medical Data Standardization Benchmark (MDS‑Bench), which evaluates vision‑language models (VLMs) on their ability to process raw, heterogeneous medical data. Models must identify source formats, convert raw images into VLM‑compatible inputs, extract relevant text, and organize the results into structured image‑text pairs. Experiments show that even the top VLM, Gemini 3 Flash, achieves only a 48.6% end‑to‑end success rate, underscoring the challenge of raw data standardization in clinical settings.

By Xin Chen, Dongliang Xu, Cunhao Zhu, Xudong Luo, Haoyang Lyu, Xiaoxiao Sun, Serena Yeung-Levy, Yue Yao

llms multimodal benchmarks

arXiv AI

Jun 9

MedVision: Benchmarking Quantitative Medical Image Analysis

arXiv:2511. 18676v2 Announce Type: replace-cross Abstract: Current vision-language models (VLMs) in medicine are primarily designed for categorical question answering (e.

By Yongcheng Yao, Yongshuo Zong, Raman Dutt, Yongxin Yang, Sotirios A Tsaftaris, Timothy Hospedales

llms nlp fine-tuning multimodal benchmarks

arXiv AI

Aug 14

Can Generalist Vision Language Models (VLMs) Rival Specialist Medical VLMs? Benchmarking and Strategic Insights

arXiv:2506. 17337v5 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) have shown promise in automating image diagnosis and interpretation in clinical settings.

By Yuan Zhong, Ruinan Jin, Qi Dou, Xiaoxiao Li

llms fine-tuning benchmarks

arXiv Computer Vision

Aug 25

MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis

arXiv:2608.22323v1 Announce Type: new Abstract: The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing benchmarks largely focus...

By Lai Wei, Yuchao Chen, Zhenbiao Cao, Xiaojin Zhang, Zhongyu Wei, Bangting Wang, Wei Chen, Xiang Bai

llms multimodal benchmarks

arXiv Machine Learning

Jul 10

A Tool Bottleneck Framework for Clinically-Informed and Interpretable Medical Image Understanding

arXiv:2512. 21414v2 Announce Type: replace-cross Abstract: Recent tool-use frameworks powered by vision-language models (VLMs) improve image understanding by grounding model predictions with specialized tools.

By Christina Liu, Alan Q. Wang, Joy Hsu, Jiajun Wu, Ehsan Adeli

llms multimodal benchmarks

arXiv AI

Sep 10

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

arXiv:2601.06847v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We po...

By Mengmeng Zhang, Xiaoping Wu, Hao Luo, Fan Wang, Yisheng Lv

llms computer-vision multimodal

arXiv Computer Vision

Aug 25

Expert-level vision-language foundation model for real-world radiology and comprehensive evaluation

arXiv:2409.16183v2 Announce Type: replace Abstract: Radiology is a vital and complex component of modern clinical workflow and covers many tasks. Recently, vision-language (VL) foundation models in m...

By Xiaohong Liu, Guoxing Yang, Yulin Luo, Jiaji Mao, Xiang Zhang, Haibo Wang, Zhiyang He, Ming Gao, Shanghang Zhang, Jun Shen, Guangyu Wang

multimodal benchmarks

arXiv AI

Jul 7

Evaluating and Understanding Model Editing for Medical Vision Language Models

arXiv:2607. 05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining.

By Guli Zhu, Chenwei Wu, Liyue Shen

llms diffusion multimodal benchmarks

arXiv AI

Jul 3

MedRepBench: A Comprehensive Benchmark for Medical Report Interpretation

arXiv:2508. 16674v2 Announce Type: replace-cross Abstract: Medical report understanding from real-world document images is essential for generating patient-facing explanations and enabling structured information exchange in clinical systems.

By Fangxin Shang, Yuan Xia, Dalu Yang, Yahui Wang, Binglin Yang

llms computer-vision multimodal benchmarks safety

arXiv AI

Jul 2

Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection

arXiv:2606. 25375v2 Announce Type: replace-cross Abstract: With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud.

By Ching-Hao Chiu, Hao-Wei Chung, Gelei Xu, Xueyang Li, Pin-Yu Chen, John Kheir, Meysam Ghaffari, Carlos Morato, Ahmed Abbasi, Yiyu Shi

llms multimodal benchmarks

arXiv AI

Aug 20

MedUAG: Unified Understanding and Generation for Medical Multimodal Models

The paper introduces MedUAG, a unified medical multimodal model that supports both understanding and generation tasks. It presents MedUAGCorpus, the largest dataset of over 6 million instances across 14 imaging modalities, and MedUAGBench, a benchmark covering 12 diverse generation tasks with standardized protocols. Experiments show that MedUAG performs strongly across many medical understanding and generation tasks, setting a competitive baseline for future medical multimodal systems.

By Zijie Meng, Yuncheng Zhang, Hualiang Wang, Yitian Tang, Xiaotang Gai, Chen Shen, Songtao Jiang, Shaosheng Cao, Jian Wu, Xian Wu, Zuozhu Liu

llms multimodal benchmarks

Hugging Face Trending Papers

Sep 1

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the sc...

llms rag multimodal benchmarks safety

Related stories