The paper introduces the Boot-and-Feedback (BooF) framework, which facilitates collaboration between multimodal large language models (MLLMs) and expert vision models for breast ultrasound (BUS) diagnosis. In the Boot Stage, the MLLM is guided by the BI-RADS lexicon and preliminary expert predictions to generate reliable textual descriptions, while the Feedback Stage fuses these descriptions with visual features using an Attention-Gated Cross-Modality Fusion Module, allowing the expert model to incorporate textual insights and filter out noise. Experiments on multiple BUS datasets show that BooF improves both diagnostic accuracy and interpretability compared to existing methods.
Breast ultrasound (BUS) is widely used for breast cancer diagnosis yet remains operator-dependent. While deep learning shows promise, ensuring diagnostic reliability and interpretability is challengin...
Breast cancer remains a leading cause of cancer-related mortality among women. Its clinical management requires multimodal reasoning across a clinical workflow that spans \textit{screening}, \textit{diagnosis} and \textit{treatment planning}, where each stage involves distinct imaging modalities, task objectives, and reasoning patterns.
The paper introduces CORAL, a multimodal framework that combines spatial grounding and concept-level supervision for medical report generation. CORAL uses a prompt-driven segmentation model to localize lesions and a Concept Bottleneck module to predict multi-class clinical attributes, feeding these textual concept tokens and mask-modulated visual features into a multimodal large language model. Experiments on BUS-CoT and IU X-ray datasets show that CORAL improves diagnostic accuracy, concept consistency, and report quality compared to existing general-purpose and medical MLLMs.
arXiv:2607. 25933v1 Announce Type: cross Abstract: Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning.
arXiv:2511. 20956v2 Announce Type: replace-cross Abstract: Breast ultrasound (BUS) reporting relies on clinically meaningful lesion descriptors, including BI-RADS category, lesion shape, margin, echogenicity, posterior features, pathology, and histology.
arXiv:2606. 28556v1 Announce Type: new Abstract: Recent advances in large language models and vision-language models have enabled reasoning over multimodal data, offering opportunities for clinical applications such as decision support and triaging.
arXiv:2608.22323v1 Announce Type: new Abstract: The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing benchmarks largely focus...
The paper introduces MedUAG, a unified medical multimodal model that supports both understanding and generation tasks. It presents MedUAGCorpus, the largest dataset of over 6 million instances across 14 imaging modalities, and MedUAGBench, a benchmark covering 12 diverse generation tasks with standardized protocols. Experiments show that MedUAG performs strongly across many medical understanding and generation tasks, setting a competitive baseline for future medical multimodal systems.
The paper introduces Generalist‑Specialist Mixture‑of‑Experts (GS‑MoE), a two‑branch architecture that combines a cross‑modal generalist model with modality‑specific specialists through domain‑constrained feature fusion. GS‑MoE improves detection of rare pathologies in multimodal medical imaging, achieving significant per‑class F1 gains and outperforming dense and specialist‑only MoE baselines while using about 53% fewer active parameters at inference. The study demonstrates that balancing cross‑modal shared representations with expert routing can enhance performance on low‑prevalence conditions.
The paper introduces SMILE, a self‑explainable multimodal information bottleneck framework for medical diagnosis. It jointly optimizes predictive accuracy and modality‑specific explainability by selecting the most informative elements within each data modality. Experiments on diverse medical datasets show strong diagnostic performance, including a 9.1‑percentage‑point accuracy gain on the iCTCF dataset, and provide transparent, modality‑aware explanations that enhance both explainability and generalization.
The paper presents a plug‑and‑play framework that uses concept-based interpretation to debug medical imaging models. By aligning a single‑modality encoder with BioMedCLIP, the authors build a Concept Bottleneck Model that allows concept‑level interventions to distinguish causal from spurious features. Experiments on Mayo Clinic ultrasound and CheXpert chest X‑ray data show that these interventions can reliably diagnose model issues and even improve performance through guided fine‑tuning.
arXiv:2603. 07131v4 Announce Type: replace-cross Abstract: Large Vision Language Models (LVLMs) show immense potential for automated ophthalmic diagnosis.