arXiv AI

Integrating knowledge from case reports: a medical ontology based multimodal information system with structured summary

The paper presents a multimodal information system that integrates structured clinical summaries, medical images, and biomedical named entities from 52,949 open‑access case reports published between 2000 and 2021. This information is organized within a well‑structured medical ontology, and a user interface is designed to help junior clinicians search and browse case reports efficiently. The system aims to improve the identification and diagnosis of rare diseases by overcoming the limitations of traditional keyword‑based retrieval on unstructured case reports.

arXiv Machine Learning
Aug 6

Neighborhood-Aware Dual Biomedical Entity Linking

arXiv:2608. 04144v1 Announce Type: cross Abstract: Biomedical entity linking grounds mentions in clinical and scientific text to entities in a curated knowledge base (KB) with ontological structure, which supports downstream applications such as literature-scale information extraction and patient-record normalization.

By Yicheng Tao, Jie Liu
arXiv AI
Jul 9

Health System Scale Semantic Search Across Unstructured Clinical Notes

arXiv:2604. 25605v2 Announce Type: replace-cross Abstract: Introduction: Semantic search, which retrieves documents based on conceptual similarity rather than keywords, offers advantages for retrieval of clinical information.

By Faith Wavinya Mutinda, Spandana Makeneni, Anna Lin, Shivaji Dutta, Irit R. Rasooly, Patrick Dibussolo, Shivani Kamath Belman, Hessam Shahriari, Kevin Murphy, Alex B. Ruan, Barbara H. Chaiyachati, Sanjay Chainani, Robert W. Grundmeier, Scott M. Haag, Jeffrey M. Miller, Heather M. Griffis, Ian M. Campbell
arXiv AI
Aug 19

A Multimodal Agentic Pathology Co-pilot via Evidence Grounded Reasoning

PathPocket is a multimodal AI co‑pilot that grounds pathology decision‑making in evidence. It builds the largest pathology evidence corpus (≈110,472 documents) and a hypergraph of 4.55 million entities and 7.10 million relations to support traceable reasoning. The system handles text and multimodal queries, including ROI and gigapixel whole‑slide images, and outperforms current state‑of‑the‑art models on a benchmark of over 200,000 real‑world cases, improving pathologists’ diagnostic accuracy and confidence.

By Zhe Xu, Zhengyu Zhang, Zhiyuan Cai, Jiahao Xu, Yijie Lin, Ziyi Liu, Junlin Hou, Hongyi Wang, Yuxiang Nie, Yihui Wang, Jiabo Ma, Ling Liang, Yingxue Xu, Zhengrui Guo, Guanghao Wu, Danyi Li, Ziqi Zhou, Donglin Tan, Zhijian Cen, Ying Tan, Xiaolin Liu, Qi Xie, Xiaoying Tang, Xi Peng, Cheng Deng, Lijuan Qu, Ronald Cheong Kin Chan, Li Liang, Hao Chen
arXiv Computation and Language
Aug 28

Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval

Case2Flow is a new task that retrieves the most relevant guideline flowchart for a given patient case from a collection of medical guideline documents. The authors created FlowAtlas, a curated corpus of 202 flowcharts extracted from 2,080 guidelines, and a pipeline that generates 1,911 aligned case‑flowchart pairs. Their evaluation shows that existing multimodal retrieval methods often overrely on keywords and spurious token‑patch matches, and they propose CRISP, a training‑free scoring method that improves Recall@1 by up to 18.71 percentage points and gains preliminary feasibility evidence from a blinded physician assessment.

By Jiale Wei, Yufan Chen, Alexander Jaus, Zdravko Marinov, Julian Friedrich, Simon Rei{\ss}, Jens Kleesiek, Rainer Stiefelhagen
arXiv Machine Learning
Sep 18

A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI

The paper introduces Biomedica, an open-source dataset sourced from PubMed Central that includes over 6 million scientific articles and 24 million image‑text pairs, along with 27 metadata fields and expert human annotations. To facilitate use, the authors provide scalable streaming and search APIs via a web server. They demonstrate the dataset’s value by training embedding models, chat‑style models, and retrieval‑augmented chat agents, all of which outperform previous open systems in their categories.

By Alejandro Lozano, Min Woo Sun, James Burgess, Jeffrey J. Nirschl, Christopher Polzak, Yuhui Zhang, Liangyu Chen, Jeffrey Gu, Ivan Lopez, Josiah Aklilu, Anita Rau, Austin Wolfgang Katzer, Collin Chiu, Orr Zohar, Xiaohan Wang, Alfred Seunghoon Song, Chiang Chia-Chun, Robert Tibshirani, Serena Yeung-Levy
arXiv AI
Aug 21

MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis

arXiv:2605. 22547v3 Announce Type: replace-cross Abstract: Medical image diagnosis has achieved significant progress with deep learning, yet existing methods often rely on isolated visual evidence and lack the ability to effectively leverage similar cases and external knowledge.

By Yiming Xu, Yixuan Liu, Yuhang Zhang, Ling Zheng, Yihan Wang, Qi Song
arXiv AI
Jul 22

Trajectory-Aware Clinical Risk Prediction via Severity-Grounded Knowledge Graphs and Retrieval-Augmented Generation

arXiv:2607. 18270v1 Announce Type: new Abstract: While Electronic Health Records (EHRs) offer a wealth of clinical data, effectively augmenting a patient's records with heterogeneous external knowledge to predict the patient's clinical risk remains a significant challenge.

By Kyunghoon Jeon, Youmin Ko, Woohwan Jung, Hyunjoon Kim
arXiv AI
Aug 11

H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System

arXiv:2608. 08056v1 Announce Type: new Abstract: Medical data, by its nature, exhibit a high degree of heterogeneity on multiple levels ranging from (a) different modalities like images, text and time series, (b) diverse tabular schemata introduced by institutions and (c) completely unstructured textual information data provided by healthcare professionals.

By Ioannis N. Tzortzis, Georgia Kapetadimitri, Agapi Davradou, Nefeli Kousta, Nikolaos Bakalos, Ioannis Rallis, Dimitrios Kalogeras, Nikolaos Doulamis, Anastasios Doulamis
Hugging Face Trending Papers
Jul 30

CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography

Large chest radiography archives are difficult to search because most studies are paired only with free-text reports rather than structured clinical annotations. Vision-language models offer a natural interface for text-to-image retrieval, but current biomedical models are primarily optimized for report-to-image matching rather than for satisfying short clinical search queries.