← Back to all news
arXiv AI By Zhichao Yang, Sepehr Janghorbani, Dongxu Zhang, Jun Han, Qian Qian, Andrew Ressler II, Gregory D. Lyng, Sanjit Singh Batra, Robert E. Tillman

Health-SCORE: Towards Scalable Rubrics for Improving Health-LLMs

Read the original on arXiv AI →

Health‑SCORE is a scalable rubric‑based framework designed to evaluate and train health‑focused large language models (LLMs). It reduces the need for extensive human expertise and development cost while maintaining evaluation quality comparable to human‑created rubrics. The framework also serves as a structured reward signal for reinforcement learning and can be embedded in prompts to enhance response quality via in‑context learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jun 17

RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills

arXiv:2606. 18203v1 Announce Type: cross Abstract: The LLM-empowered personal health agents with user health (sensor) metrics have offered a promising pathway to alleviate global disparities in healthcare access.

By Weizhi Zhang, Zechen Li, Hamid Palangi, Ben Graef, A. Ali Heydari, Simon A. Lee, Salman Rahman, Ray Luo, Zeinab Esmaeilpour, Erik Schenck, Chloe Zhang, Yamin Li, Menglian Zhou, Philip S. Yu, Daniel McDuff, Lindsey Sunden, Mark Malhotra, Shwetak Patel, Ahmed A. Metwally
llmsagentsefficiencysafety
More like this →
arXiv AI
Oct 31

Rubricon: Automated Rubric Generation and Refinement for LLM Evaluation and Training

Rubricon is a pipeline that automatically generates and refines discriminative rubrics for evaluating large language models (LLMs) without requiring human preference labels or manual rubric creation. It samples multiple rubrics, selects the most discriminative ones, adds missing criteria, prunes misaligned or shallow criteria, and uses LLM self‑critique and observed pass rates to guide refinement. Across diverse LLM benchmarks, Rubricon’s rubrics achieve 9% better alignment with expert‑authored rubrics than other synthetic methods, and training a Qwen3.5‑9B model on HealthBench with Rubricon rubrics improves performance by 4.5% over the best alternative synthetic generator.

By Ulyana Tkachenko, Hui Wen Goh, Guram Gogia, Jonas Mueller
llmsagentsbenchmarkssafety
More like this →