arXiv AI
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
3d ago
A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses
arXiv:2609.37788v1 Announce Type: cross Abstract: Rubrics support the structured evaluation of language models. We propose a rubric for assessing expressed clinical reasoning in model responses, draw...
arXiv AI
Sep 15
Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment
The paper introduces a paired benchmark to detect hindsight bias in clinical language models by comparing model responses to questions posed at a clinically relevant cutoff versus the full timeline. It uses 171 case reports (40 sepsis, 131 GLP‑1/diabetes) with both human‑annotated and LLM‑generated time‑series data, evaluating accuracy, hindsight trap rate, answer instability rate, and hindsight bias rate. Results show that exposing models to the full timeline consistently increases hindsight bias, while truncating the timeline mitigates bias without sacrificing accuracy.
arXiv AI
Jul 16
Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
arXiv:2607. 13036v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for decision support in healthcare, but clinical evidence is often incomplete or evolving.
arXiv AI
Aug 26
STRIVE: Multi-Agent Structured Temporal Reasoning with Integrated Verification for Longitudinal Radiology Report Generation
STRIVE is a new framework for longitudinal radiology report generation that separates clinical reasoning into distinct Diagnosis, Attribute, and Temporal Change agents, each producing explicit evidence. The Temporal Change agent is refined with a Progression-Aware GRPO reward that differentiates direction-preserving errors from reversals. Verification occurs twice: a Consistency Gate aligns agent outputs before report generation, and a Validation Agent ensures the final report is supported by the aggregated evidence. On the Longitudinal-MIMIC dataset, STRIVE achieves superior clinical efficacy and more than doubles Longitudinal Change Concordance compared to the strongest baseline.
arXiv AI
Aug 25
Teaching LLMs How ICU Physicians Approach Clinical Reasoning Through OMOP-Aligned Retrieval Improves Reasoning Across Clinical Domains
arXiv:2608.22622v1 Announce Type: cross Abstract: Clinical decision-making relies on identifying relevant patient information to guide diagnosis and treatment, a challenge that is especially difficul...
arXiv AI
Aug 3
Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support
arXiv:2607. 28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning.