arXiv AI By Misaki Matsuura, Sayantan Kumar, Ojas Kadam, Jeremy C. Weiss

Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment

Read the original on arXiv AI →

The paper introduces a paired benchmark to detect hindsight bias in clinical language models by comparing model responses to questions posed at a clinically relevant cutoff versus the full timeline. It uses 171 case reports (40 sepsis, 131 GLP‑1/diabetes) with both human‑annotated and LLM‑generated time‑series data, evaluating accuracy, hindsight trap rate, answer instability rate, and hindsight bias rate. Results show that exposing models to the full timeline consistently increases hindsight bias, while truncating the timeline mitigates bias without sacrificing accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 2

A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined

The article reviews existing tools for assessing clinical reasoning in large language models, identifying six key dimensions—problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. It finds that no single instrument covers all dimensions, with some aspects reasonably addressed while others, especially faithfulness and counterfactual reasoning, remain weakly evaluated. The authors propose combining current tools through binary rubric items, separate completeness and correctness scores, importance weighting, safety caps, temporal consistency checks, and chance‑corrected reliability reporting, and call for further design work on uncertainty, counterfactual reasoning, and faithfulness in longitudinal free‑text contexts.

By Zhangshu Joshua Jiang, Zina Ibrahim, James T. Teo
arXiv AI
3d ago

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

The paper presents a method for reconstructing clinical timelines by extracting structured evidence for narrative events and jointly revising their estimated times, while preserving unique identifiers linked to source spans. It introduces GAVEL, a large language model judge that verifies disagreements between timelines against notes and structured records, citing evidence for its decisions. Evaluation on clinician-annotated discharge summaries shows that adding structured evidence improves timestamp agreement for some models, and GAVEL consistently favors multimodal timelines over text-only ones.

By Sayantan Kumar, Nicolas Grimaldi, Jack Cummins, Jeremy C. Weiss
arXiv AI
Jun 9

TRIAGE: Dialectical Reasoning for Explainable Risk Prediction on Irregularly Sampled Medical Time Series with LLMs

arXiv:2606. 09030v1 Announce Type: cross Abstract: Clinical early warning systems built on electronic health records, in which clinical observations are recorded as irregularly sampled medical time series (ISMTS), must deliver both calibrated risk scores for patient triage and interpretable rationales that clinicians can verify.

By Hyeongwon Jang, Gyouk Chu, Changhun Kim, Joonhyung Park, Hangyul Yoon, Eunho Yang