Health‑SCORE is a scalable rubric‑based framework designed to evaluate and train health‑focused large language models (LLMs). It reduces the need for extensive human expertise and development cost while maintaining evaluation quality comparable to human‑created rubrics. The framework also serves as a structured reward signal for reinforcement learning and can be embedded in prompts to enhance response quality via in‑context learning.
arXiv:2609.24480v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) in healthcare requires robust performance across two complementary dimensions - diagnostic reasoning: the conv...
arXiv:2606. 05174v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown strong promise in healthcare applications.
arXiv:2606. 18203v1 Announce Type: cross Abstract: The LLM-empowered personal health agents with user health (sensor) metrics have offered a promising pathway to alleviate global disparities in healthcare access.
Despite rapid advances in large language models (LLMs), their deployment in the sports and health domain remains high-risk due to the absence of domain-specific evaluation benchmarks. Existing general...
Rubricon is a pipeline that automatically generates and refines discriminative rubrics for evaluating large language models (LLMs) without requiring human preference labels or manual rubric creation. It samples multiple rubrics, selects the most discriminative ones, adds missing criteria, prunes misaligned or shallow criteria, and uses LLM self‑critique and observed pass rates to guide refinement. Across diverse LLM benchmarks, Rubricon’s rubrics achieve 9% better alignment with expert‑authored rubrics than other synthetic methods, and training a Qwen3.5‑9B model on HealthBench with Rubricon rubrics improves performance by 4.5% over the best alternative synthetic generator.
arXiv:2610.26185v2 Announce Type: replace Abstract: Despite rapid advances in large language models (LLMs), their deployment in the sports and health domain remains high-risk due to the absence of do...
The paper introduces TIR-EC, a tool‑integrated reinforcement learning framework designed to improve large language models (LLMs) for emergency department prediction tasks. TIR-EC exposes groups of electronic health record (EHR) features as callable tools and trains the LLM using rewards from a gradient‑boosted tree teacher, while a reverse curriculum gradually removes prompt‑based skills to embed task‑specific clinical knowledge into the model weights. Experiments on three clinical prediction tasks show that a 4B TIR‑EC agent achieves the highest macro F1 among compared models, surpasses conventional machine‑learning baselines, and competes with much larger frontier LLMs, while also selecting more useful evidence with fewer tool calls.
arXiv:2412.15957v2 Announce Type: replace-cross Abstract: The rapid development of large language models (LLMs) has transformed many industries, including healthcare. In practice, hospitals and patie...
arXiv:2610.02824v1 Announce Type: new Abstract: Rubric-based reinforcement learning extends reward-driven optimization to open-ended tasks by assigning partial credit to individual response requireme...
arXiv:2610.17733v1 Announce Type: cross Abstract: Rubric-based reinforcement learning (RL) evaluates responses against explicit criteria to construct rewards, offering a viable way to train large lan...
Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks. Rubric-based judges offer a transparent way to decompose such judgments into explicit evaluation criteria, but existing annotation-free rubric generators typically rely on a single generic evaluator.
EvoRubric introduces a co‑evolutionary reinforcement learning framework that learns rubric‑driven evaluation criteria for open‑ended text generation. The system uses a shared policy that simultaneously generates responses and rubrics, while a frozen meta‑verifier and grader provide validity checks and scoring. Experiments on medical, writing, and science benchmarks show that EvoRubric outperforms static and other evolving‑rubric baselines, achieving higher average scores on five tasks.