arXiv:2607. 02879v1 Announce Type: new Abstract: Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query.
By Siran Zhao, Ruihui Hou, Ziyue Huai, Chennuo Zhang, Tong Ruan
arXiv:2605. 30295v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise for clinical reasoning and decision support, but evaluation in realistic, electronic health record-congruent settings remains limited.
By Valentina Bui Muti, Eug\'enie Dulout, Ziquan Fu
arXiv:2601. 23049v2 Announce Type: replace Abstract: Medical calculators are fundamental to quantitative, evidence-based clinical practice.
By Yakun Zhu, Yutong Huang, Shengqian Qin, Zhongzhen Huang, Shaoting Zhang, Xiaofan Zhang
The paper introduces a retrieval‑augmented multi‑agent framework that automatically generates instance‑specific evaluation rubrics for medical language models. By retrieving authoritative medical evidence, decomposing it into atomic facts, and combining these with user interaction constraints, the system produces fine‑grained criteria that outperform GPT‑4o on HealthBench and LLMEval‑Med. The generated rubrics also guide response refinement, improving medical LLM output quality by 9.2%.
By Yinzhu Chen, Abdine Maiga, Hossein A. Rahmani, Emine Yilmaz
arXiv:2608. 16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation.
By Yifan Zhang, Rahmatollah Beheshti
The paper explores a new approach to improve arithmetic reliability in clinical language models by having the model generate case‑specific Python code that a local executor runs deterministically, rather than performing arithmetic directly. Experiments on the MedCalc‑Bench Verified dataset show that this Program‑Solve interface yields modest gains for larger models (Qwen2.5‑32B) but not for smaller ones (Qwen2.5‑7B), and it does not replace the need for verified formulas or accurate variable extraction. The study highlights that adding an executor can help some open‑weight models but is not a universal solution.
By Felipe Ocampo Osorio, Sebasti\'an Andr\'es Cajas Ordo\~nez, Maximin Lange, Rafi Al Attrach, Sahil Kapadia, Zakaria Laouabdia Sellami, Angelo Antonio Talio, Leo Anthony Celi