arXiv Computation and Language

MedicalBench: Evaluating Large Language Models Toward Improved Medical Concept Extraction

arXiv AI
Jul 28

Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing

arXiv:2607. 23368v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are demonstrating significant capabilities in medical tasks like radiology analysis, yet providing faithful and interpretable explanations remains a key consideration for their responsible deployment in clinical settings.

By Jakub Rymarski (University of Warsaw, Poland), Adam Rempa{\l}a (University of Warsaw, Poland), Bart{\l}omiej Sobieski (University of Warsaw, Poland), Przemys{\l}aw Biecek (University of Warsaw, Poland)
arXiv AI
Aug 18

DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models

arXiv:2505. 14107v5 Announce Type: replace-cross Abstract: The emergence of groundbreaking large language models capable of performing complex reasoning tasks holds significant promise for addressing various scientific challenges, including those arising in complex clinical scenarios.

By Yakun Zhu, Zhongzhen Huang, Linjie Mu, Yutong Huang, Wei Nie, Jiaji Liu, Shaoting Zhang, Pengfei Liu, Xiaofan Zhang
arXiv Computation and Language
Aug 31

UIC-AIHealth4All at ArchEHR-QA 2026: Answer-First Evidence Grounding for Clinical Question Answering

The UIC-AIHealth4All system was presented for the ArchEHR-QA 2026 shared task on grounded question answering from electronic health records. It participated in evidence identification, answer generation, and answer‑evidence alignment, using an answer‑first pipeline that generates candidate answers with cited note sentences before classifying the full evidence set. The system ranked third in evidence identification, ninth in answer generation, and fifth in answer‑evidence alignment, and a linguistic analysis showed its outputs were harder to read than clinician‑authored references, highlighting the need for readability optimization in clinical NLP.

By Mohammad Arvan, Hossein Haeri, Natalie Parde, Rebecca T. Feinstein
arXiv Computation and Language
Aug 27

Retrieval-Augmented Agentic Rubric Generation for Reliable Medical Response Evaluation

The paper introduces a retrieval‑augmented multi‑agent framework that automatically generates instance‑specific evaluation rubrics for medical language models. By retrieving authoritative medical evidence, decomposing it into atomic facts, and combining these with user interaction constraints, the system produces fine‑grained criteria that outperform GPT‑4o on HealthBench and LLMEval‑Med. The generated rubrics also guide response refinement, improving medical LLM output quality by 9.2%.

By Yinzhu Chen, Abdine Maiga, Hossein A. Rahmani, Emine Yilmaz
arXiv AI
Sep 10

Building evidence-based knowledge bases from full-text literature for disease-specific biomedical reasoning

EvidenceNet is a disease‑specific dataset that transforms full‑text biomedical literature into structured evidence records and graph representations, preserving study design, provenance, and quantitative support. Using an LLM‑assisted pipeline, it extracts experimentally grounded findings, normalizes entities, scores evidence quality, and links related records via typed semantic relations. The released subsets—EvidenceNet‑HCC and EvidenceNet‑CRC—contain thousands of evidence records and richly connected graphs, with high extraction and relation‑type accuracy, enabling retrieval‑augmented question answering and graph‑based tasks such as link prediction and target prioritization.

By Chang Zong, Jinyu Chen, Sicheng Lv, Si-tu Xue, Huilin Zheng, Jian Wan, Lei Zhang
arXiv AI
Oct 2

A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classification

The paper introduces a comparative explainability framework for auditing DeBERTa‑v3 in zero‑shot medical abstract classification. It evaluates five explanation methods—SHAP, LIME, occlusion, Input × Gradient, and Attention × Gradient—using a natural language inference engine on a balanced corpus of 1,000 abstracts per diagnostic category. The study finds that explanatory stability aligns with predictive certainty, identifies three systemic failure mechanisms, and recommends combining multiple explanation methods and quantitative agreement metrics for transformer‑based medical text classifiers.

By Javier Diaz Esteban-Herreros, David Mu\~noz-Valero, Raquel Mart\'inez-Espa\~na, Jose M. Juarez, Juan Moreno-Garcia