arXiv Computer Vision

Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models

arXiv Computation and Language
Oct 12

CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

CliQ‑RRG is a two‑stage framework for generating radiology reports that aligns visual and textual data using disease‑aware embeddings and prior‑guided attention to capture multi‑view dependencies. The second stage retrieves graph‑grounded clinical evidence and employs a large language model to produce concise, interpretable question‑answer style reports with diagnostic summaries. Experiments on two chest X‑ray benchmarks show that CliQ‑RRG outperforms previous methods on both lexical and clinical metrics, producing accurate and clinically coherent reports.

By Chandravardhan Singh Raghaw, Shahid Shafi Dar, Arnav Nishith Sharma, Shrey Modi, Nagendra Kumar
Hugging Face Trending Papers
Oct 9

CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

CliQ‑RRG is a two‑stage framework for generating radiology reports that aligns visual and textual data using disease‑aware embeddings and a Prior‑Guided Attention Module to capture multi‑view dependencies. In the second stage, GraphRAG‑guided retrieval adds graph‑grounded clinical evidence, and a large language model restructures the content into concise question‑answer pairs with diagnostic summaries. Experiments on two chest X‑ray benchmarks show that CliQ‑RRG outperforms previous methods in both lexical and clinical metrics, producing accurate and clinically coherent QA‑style reports.

Hugging Face Trending Papers
Jun 24

Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography

Vision-language pre-training (VLP) holds great promise for general-purpose medical AI by leveraging radiology reports as rich textual supervision, yet existing methods struggle with 3D CT imaging due to inefficient visual backbones and coarse semantic alignment. To address these issues, we propose a tailored VLP framework featuring three key components: (1) a CNN-ViT hybrid encoder that replaces ViT's patch embedding with a 3D CNN backbone to efficiently capture local anatomical details while preserving global attention and compatibility with pre-trained cross-modal priors; (2) a disease-level contrastive learning mechanism using learnable query tokens to dynamically extract disease-specific semantics from full reports and align them with corresponding visual features, thereby disentangling distinct diseases within the same anatomical region; and (3) a diagnosis-aware prompt strategy that employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap and enhance zero-shot diagnostic reliability.

arXiv Computation and Language
Aug 28

Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models

The paper investigates how to incorporate biomedical knowledge graphs (KGs) into large language models (LLMs) for clinical diagnosis. It evaluates five KG task formulations, three training paradigms, two KGs, and three base LLMs, finding that all paradigms outperform a non‑finetuned baseline but differ in knowledge transfer behavior. Introducing Gradient Intervention Density (GID) and Gradient Distortion (GD) metrics, the study identifies a ‘surgical alignment’ regime—sparse, localized updates achieved by KG‑judgment training with KL regularization—that improves reasoning quality even when in‑domain accuracy is lower than task‑specific supervised fine‑tuning.

By Saksham Khatwani, He Cheng, Majid Afshar, Dmitriy Dligach, Yanjun Gao
arXiv AI
Aug 5

CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement

arXiv:2608. 03890v1 Announce Type: cross Abstract: A clinically useful chest X-ray system must go beyond fluent report generation: it should classify findings with tunable decision thresholds, localize them spatially, and derive the anatomical measurements upon which many diagnoses depend.

By Mercy Prasanna Ranjit, Anirban Porya, Sathvik Joel, Niharika Vadlamudi, Nikhilesh Chowdary Eathamukkala, Prasanth V V, Abhyuday Kumara Swamy, Pranay Narhari Umredkar, Pradeep Narayan, Vivek Rajagopal, Tanuja Ganu
arXiv AI
Sep 4

NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis

NeoRed is a multimodal large language model specifically designed for diagnosing neonatal respiratory diseases. It addresses two main limitations of existing models—domain gaps from adult data and inadequate integration of clinical context—by leveraging two real-world neonatal datasets (NeoCXR and NeoCXR-EV). The model incorporates a Knowledge-Logic-Alignment framework that injects diagnostic priors, aligns report semantics with diagnostic logic, and aligns visual features with imaging conclusions, achieving superior performance on neonatal benchmarks while maintaining adult benchmark performance.

By Yinan Liu, Hongtai Xia, Haoran Xu, Jiankang Hong, Jingkuan Song, Ye Luo
arXiv AI
Sep 18

FOCAL: Fine-Grained Optimal-Transport-Driven Contrastive Alignment of Language and ECGs with Waveform Enhancement

FOCAL is a framework that aligns fine-grained ECG waveform segments with specific report tags using Optimal Transport, addressing the lack of localized representation in prior methods. It introduces a semantic similarity matrix to mitigate false negatives when reports share diagnoses, and a coarse‑to‑fine enrichment pipeline that employs Large Language Models to recover missing waveform semantics while filtering hallucinations. Experiments on six datasets show FOCAL achieves state‑of‑the‑art zero‑shot prediction and linear probing performance.

By Haitao Li, Che Liu, Zhengyao Ding, Ziyi Liu, Wenqi Shao, Zhengxing Huang