The paper introduces Cros, a risk‑constrained stopping layer for sequential clinical diagnosis agents that determines when to stop testing and make a diagnosis. Cros combines state‑wise error ranking, policy design on disjoint development splits, and exact tests of selective diagnostic error to provide finite‑sample guarantees. On a MIMIC‑derived abdominal‑pain benchmark, Cros achieves higher state‑error AUROC and lower selective error rates compared to baseline stopping methods, though its performance varies across development resplits.
arXiv:2608.23397v1 Announce Type: new Abstract: Interactive clinical agents must gather decisive evidence and convert it into grounded actions under partial observability. A correct final diagnosis a...
arXiv:2609.13543v1 Announce Type: new Abstract: LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different...
The paper introduces a benchmark of 400 synthetic healthcare workflow episodes to evaluate large language model (LLM) agents’ reliability in escalating care when evidence is missing or conflicting. It finds that while clean decision accuracy is high (0.97), models miss a large proportion of necessary escalations (32–97%) under uncertainty, and even the largest 9.7B model’s performance drops to 0.78 in conflict scenarios. A workflow-aware controller improves utility and reduces missed escalations, but its effectiveness is limited by lexical generalization failures.
arXiv:2605. 12895v2 Announce Type: replace-cross Abstract: Clinical decision-support systems are expert systems whose recommendations clinicians act on directly, yet they are usually cleared on one aggregate accuracy number from a held-out test set.
arXiv:2607. 18828v1 Announce Type: new Abstract: Readiness stress-testing of medical AI has focused on closed-ended and multimodal benchmarks.
arXiv:2607. 28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning.
arXiv:2603. 24481v2 Announce Type: replace Abstract: Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings.
Predicting early outcomes based on trajectory can decrease the expenses associated with agent evaluation by terminating a run once the outcome becomes sufficiently predictable, assuming that the predi...
arXiv:2607. 28608v1 Announce Type: new Abstract: Clinical risk models routinely achieve strong aggregate performance while producing materially different error rates across patient subgroups.
The paper introduces Evidence-Carrying Termination (ECT), a method that allows tool‑using large language model agents to declare a task complete only when a typed certificate links every required answer claim to valid, in‑scope trace evidence and a deterministic replay confirms the claimed value. In controlled experiments across 48 synthetic tasks and 576 trajectories, ECT eliminated unsafe completions and premature unsupported terminations, outperforming existing termination critics and controllers while maintaining comparable supported completion rates.
arXiv:2610.07570v1 Announce Type: new Abstract: In clinical practice, agreement among independent experts is treated as evidence of reliability, and multi-round consensus has become a core mechanism...
The paper introduces a new evaluation method called "same-input rerun" to assess the consistency of clinical language‑model agents across repeated runs. By replaying 1,000 MedAgentBench tasks with identical inputs, the authors find that action‑level outputs—such as test orders, medication requests, and referrals—vary significantly, even when benchmark scores remain unchanged. The study demonstrates that current benchmarks, which typically evaluate only a single run per task, can miss substantial behavioral divergence.