arXiv Computer Vision

Multimodal Generative AI Assists Comprehensive Echocardiographic Interpretation

EchoChat is a generative multimodal vision‑language model that performs comprehensive echocardiographic analysis, emulating a cardiologist’s cognitive workflow. Trained on 46,358 examinations, it achieves state‑of‑the‑art performance across 33 cardiac conditions, including 15 new ones, and outperforms existing foundation models in both recognition and quantification tasks. In a blinded reader study, reports co‑generated with EchoChat were rated higher in diagnostic accuracy and completeness, improving F1 score from 72.78% to 82.75%.

arXiv AI
Sep 15

MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management

arXiv:2603.22179v2 Announce Type: replace Abstract: Cardiovascular disease remains the leading cause of global mortality, with progress hindered by human interpretation of complex cardiac tests. Curr...

By Jack W O'Sullivan, Mohammad Asadi, Lennart Elbe, Akshay Chaudhari, Tahoura Nedaee, Francois Haddad, Ivan Lopez, Fang Cao, Michael Salerno, Li Fe-Fei, Ehsan Adeli, Rima Arnaout, Euan A Ashley
arXiv AI
Jul 24

Enhancing Explainable Cardiac Diagnosis with Guide-Grounded Multimodal LLMs

arXiv:2607. 20814v1 Announce Type: new Abstract: The electrocardiogram (ECG) is a cornerstone of cardiac as- sessment, yet clinical deployment of deep learning models remains con- strained by limited interpretability and the hallucination risk of large language models (LLMs).

By Hai-Nam Duy Vuong, Duy-Anh Bui, Trong-Nghia Nguyen, Kim-Ngan Thi Nguyen, Trang Mai Xuan, Tien-Cuong Nguyen, Van-Dem Pham, Thien Van Luong
arXiv Machine Learning
Oct 12

EchoTab: Learning Measurement-Grounded Representations for Echocardiographic Diagnosis through Tabular Reconstruction

EchoTab is a representation learning framework for multi‑view echocardiography that uses measurement‑specific tokens as an intermediate representation between echo views and diagnosis. Each token attends to the views relevant to its measurement and is trained via tabular reconstruction to predict the measurement value, producing a factorized representation that encodes clinically meaningful structural and functional properties. Experiments on two datasets show that EchoTab improves diagnosis classification and that its learned attention aligns with cardiologists’ established measurement‑view relationships.

By Michelle Espranita Liman, Ioana Ionac, Alexander M\"uller, Eimo Martens, Daniel Rueckert, Philip M\"uller
arXiv AI
Jun 2

CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations

arXiv:2606. 00123v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance on public medical benchmarks, yet existing evaluations often remain weak proxies for clinical use, relying on isolated inputs and simplified recognition-style tasks.

By Zixian Su, Hongkai Zhang, Fan Gao, Encheng Su, Taiping Qu, Jingwei Guo, Nan Zhang, Hui Wang, Zhen Zhou, Kairui Bo, Yan Chen, Yue Ren, Shuai Li, Lei Xu, Henggui Zhang
arXiv AI
Oct 31

Integrating Heterogeneous Echocardiography AI Outputs in Language Space: Learnable Late Fusion with a Supervised LLM

The paper presents a supervised large language model (LLM) that performs late fusion of heterogeneous echocardiography AI outputs directly in language space. By aggregating view‑specific predicted findings and measurements, the LLM learns a case‑level fusion function without requiring conversion to binary decisions or feature engineering. In a patient‑level three‑fold cross‑validation on 859 examinations, the method matched the best reference strategy and surpassed a general‑purpose LLM that lacked task‑specific training.

By Ryo Takizawa, Tempei Kabayama, Chiemi Yamazaki, Shun Kitamura, Ryo Matsuoka, Yuta Ando, Satoshi Kodera, Norihiko Takeda
arXiv AI
Jun 16

XMedFusion: A Knowledge-Guided Multimodal Perception and Reasoning Framework for Autonomous Medical Systems

arXiv:2606. 14766v1 Announce Type: cross Abstract: Autonomous medical and robotic systems increasingly rely on intelligent perception and reasoning capabilities to interpret visual data and support clinical decision making.

By Hamza Riaz, Arham Haroon, Maha Baig, Muhammad Dawood Rizwan, Muhammad Naseer Bajwa, Muhammad Moazam Fraz
arXiv AI
Sep 24

NV-Reason-CT: 3D Visual Language Model for CT Analysis

NV-Reason-CT is a generative vision‑language model designed for chest and abdominal CT analysis that preserves native 3D visual encoding and incorporates radiologist‑guided reasoning. The system couples a 3D vision transformer with a language model, feeding all visual tokens and their 3D coordinates directly into language decoding to maintain volumetric spatial information. Trained on a curated corpus of about 550,000 multimodal instruction examples, the model supports abnormality classification, report generation, and interactive reasoning, achieving strong performance on CT benchmarks and reducing expert interpretation time by 50%.

By Andriy Myronenko, Dong Yang, Yucheng Tang, Baris Turkbey, Benjamin Simon, Stephanie Harmon, Rikhil Makwana, Mariam Aboian, Sena Azamat, Ibrahim Ethem Hamamci, Sezgin Er, Bjoern Menze, Marc Edgar, Yufan He, Pengfei Guo, Daguang Xu
arXiv AI
Sep 10

A radiographic world model for clinical reasoning and evidence generation

The paper introduces MedDream, a radiographic world model that learns a shared continuous latent state from paired chest radiograph-text observations. MedDream outperforms existing diagnostic and generative AI models across eight clinical datasets, improving diagnostic reasoning, resident concordance, and evidence generation. Targeted synthetic augmentation guided by subgroup performance gaps further enhances model performance, particularly for Asian patients.

By Suyang Xi, Songtao Hu, Shansong Wang, Mojtaba Safari, Luke del Balzo, Ehsan Ul Karim, Mingzhe Hu, Kuo Zhang, Tonghe Wang, Ralph R. Weichselbaum, Xiaofeng Yang