arXiv AI
MedHarnessBench: A Unified Benchmark for Healthcare Agent Evaluation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 1
CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?
arXiv:2605.16679v3 Announce Type: replace-cross Abstract: End-to-end automation of realistic healthcare operations stresses three capabilities underrepresented in current benchmarks: policy density,...
arXiv AI
Sep 15
Asclepius: An Adaptive Harness for Long-Horizon Clinical Agents
arXiv:2609.13543v1 Announce Type: new Abstract: LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different...
arXiv AI
Aug 25
MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction
arXiv:2608.23397v1 Announce Type: new Abstract: Interactive clinical agents must gather decisive evidence and convert it into grounded actions under partial observability. A correct final diagnosis a...
arXiv AI
Jun 11
MedCTA: A Benchmark for Clinical Tool Agents
arXiv:2606. 11702v1 Announce Type: cross Abstract: To make clinically grounded decisions, medical AI agents are expected to go beyond simple recognition and be capable of tool retrieval, evidence acquisition, and integration.
arXiv AI
3d ago
Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures
The paper introduces a benchmark of 400 synthetic healthcare workflow episodes to evaluate large language model (LLM) agents’ reliability in escalating care when evidence is missing or conflicting. It finds that while clean decision accuracy is high (0.97), models miss a large proportion of necessary escalations (32–97%) under uncertainty, and even the largest 9.7B model’s performance drops to 0.78 in conflict scenarios. A workflow-aware controller improves utility and reduces missed escalations, but its effectiveness is limited by lexical generalization failures.
arXiv AI
Jun 2
AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
arXiv:2606. 01961v1 Announce Type: new Abstract: Autonomous agents are increasingly expected to support end-to-end medical-AI research workflows, moving beyond isolated prediction tasks or short-form clinical question answering.