GPAgentBench-2K is a new benchmark that evaluates large language model agents in primary‑care clinical decision‑making using a constrained Markov Decision Process (CMDP). It is built from expert‑validated records of real GP encounters and includes six foundational clinical actions, a workflow prior, and safety‑informed abstention as an outcome. Testing 16 state‑of‑the‑art LLMs shows that performance drops as the action space grows and that even the best models violate safety constraints in more than half of high‑risk cases, highlighting a gap between clinical quality and safety.
arXiv:2606. 00027v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed across healthcare, yet existing benchmarks fail to capture model behavior under adversarial or ethically complex conditions common in clinical practice.
arXiv:2607. 25485v1 Announce Type: new Abstract: Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf.
arXiv:2606. 11702v1 Announce Type: cross Abstract: To make clinically grounded decisions, medical AI agents are expected to go beyond simple recognition and be capable of tool retrieval, evidence acquisition, and integration.
arXiv:2606. 31179v1 Announce Type: new Abstract: As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications.
arXiv:2606. 14149v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare settings, yet their tendency to hallucinate poses risks when clinical decisions are involved.
arXiv:2601. 22324v3 Announce Type: replace Abstract: Modern clinical practice relies on evidence-based guidelines implemented as compact scoring systems composed of a small number of interpretable decision rules.
As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment.
arXiv:2609.00296v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly proposed for healthcare tasks such as clinical documentation, evidence retrieval, patient messaging,...
arXiv:2510. 17532v2 Announce Type: replace-cross Abstract: Predicting cancer treatment outcomes requires models that are both accurate and interpretable, particularly in the presence of heterogeneous clinical data.
arXiv:2609.13543v1 Announce Type: new Abstract: LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different...
arXiv:2606. 17405v1 Announce Type: new Abstract: Clinical decision support AI systems (CDSASs) must adapt to evolving patient conditions in real-time while adhering to strict safety constraints.
The paper introduces AGVF, a multi‑agent framework for generating medical‑necessity appeals that must adhere to payer policy and evidence constraints. AGVF treats appeal synthesis as a Constrained Markov Decision Process involving five agents—policy formalization, evidence retrieval, gap analysis, adversarial critique, and gated synthesis—and proves that refining the policy constraint graph monotonically reduces evidence deficiency. A deterministic citation‑grounding gate ensures no unsupported assertions enter the shared state, and validation on 1,000 synthetic cases shows zero citation violations and monotonic deficiency reduction.