BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.
By Chen Shen
arXiv:2609.24372v1 Announce Type: new
Abstract: In-context learning (ICL) based on large language models (LLMs) has shown promising potential in alleviating performance bottlenecks caused by the limi...
By Jingyu Wang, Shijie Wu, Fusheng Jin
The paper introduces Constrained Entity Selection under Partial Knowledge (CES-PK), a framework for improving large language model (LLM) based knowledge graph question answering (KGQA) by filtering candidate answers with lightweight symbolic constraints instead of full semantic parsing. CES-PK uses a three-valued constraint semantics—satisfied, violated, unknown—to handle incomplete knowledge graphs and avoid incorrect rejections under open‑world assumptions. Experiments on the Hetionet biomedical knowledge graph show that applying type, relation, and exclusion constraints increases precision while preserving recall, and that satisfied constraints can be used to rank remaining candidates.
By Emanuel Kitzelmann
arXiv:2609.35549v2 Announce Type: replace
Abstract: Rare-disease diagnosis is a long-tail reasoning problem: phenotypes are incomplete, individual disorders are sparsely documented, and relevant evid...
By Bo Zhang, Yuchen Wang, Dongbai Li, Matthew Yu Heng Wong, Qingkai Zeng, Lijun Wang, Tien-Yin Wong, Peng Cui, Tianyu Liu
arXiv:2604. 25605v2 Announce Type: replace-cross Abstract: Introduction: Semantic search, which retrieves documents based on conceptual similarity rather than keywords, offers advantages for retrieval of clinical information.
By Faith Wavinya Mutinda, Spandana Makeneni, Anna Lin, Shivaji Dutta, Irit R. Rasooly, Patrick Dibussolo, Shivani Kamath Belman, Hessam Shahriari, Kevin Murphy, Alex B. Ruan, Barbara H. Chaiyachati, Sanjay Chainani, Robert W. Grundmeier, Scott M. Haag, Jeffrey M. Miller, Heather M. Griffis, Ian M. Campbell
arXiv:2609.08174v1 Announce Type: new
Abstract: We introduce OntologyBench, a tiered biomedical retrieval benchmark comprising 471,854 training and 125,744 evaluation query-document relevance pairs a...
By Xiao Yu Cindy Zhang, Wyeth Wasserman, Jian Zhu
arXiv:2606. 23992v1 Announce Type: cross Abstract: Clinical value sets define the standardized terminology codes used in quality measurement, phenotyping, cohort construction, and clinical decision support.
By Sumit Mukherjee
The paper introduces a retrieval‑augmented multi‑agent framework that automatically generates instance‑specific evaluation rubrics for medical language models. By retrieving authoritative medical evidence, decomposing it into atomic facts, and combining these with user interaction constraints, the system produces fine‑grained criteria that outperform GPT‑4o on HealthBench and LLMEval‑Med. The generated rubrics also guide response refinement, improving medical LLM output quality by 9.2%.
By Yinzhu Chen, Abdine Maiga, Hossein A. Rahmani, Emine Yilmaz
The paper introduces the concept of LLM‑specific utility, defining it as the performance gain a target large language model (LLM) achieves when provided with a passage compared to answering without evidence. A benchmark of utilitarian passages is built for four LLMs (Qwen3‑8B/14B/32B and Llama 3.1‑8B) across three QA datasets, revealing that each model benefits most from its own tailored evidence and that evidence optimized for other models is consistently suboptimal. The authors also create SpecUBench, a benchmark for LLM‑specific utility judgment, and show that current utility‑aware retrieval methods largely capture model‑agnostic usefulness, struggling to estimate LLM‑specific utility.
"whyItMatters":"The study demonstrates that retrieval‑augmented generation must consider model‑specific evidence selection to truly improve LLM performance, highlighting a gap in existing utility‑aware methods."
By Hengran Zhang, Keping Bi, Jiafeng Guo, Jiaming Zhang, Shuaiqiang Wang, Dawei Yin, Xueqi Cheng
The paper presents a method for extracting key information from OCR‑digitized clinical reports, addressing challenges posed by heterogeneous documents and noisy OCR output. It introduces an open key space that is iteratively mined, normalized, clustered, and verified to build a canonical key inventory, and defines key coverage as a metric for inventory completeness. Experiments on reports from over 20 hospitals using a 0.2B BERT model show that performance improves steadily with key coverage, achieving high F1 scores when the top 90 keys are covered and outperforming a fine‑tuned Qwen3‑0.6B baseline.
By Yu Wang, Yingyun Li, Ying Qin, Haiyang Qian
arXiv:2609.36392v1 Announce Type: new
Abstract: In diseases where clinical guidelines are incomplete, contested, or mutually contradictory, knowledge completeness and dynamic conflict-aware synthesis...
By Yuyan Chen
The paper introduces Tasks over Application Manuals (TAM), a benchmark designed to test long‑horizon procedural reasoning in large language models. TAM uses real‑world tasks from ICD‑10‑CM clinical coding and U.S. federal sentencing, requiring models to follow extensive, rule‑based manuals and perform interdependent steps to produce exact answers. Experiments with GPT‑5 and various prompting strategies show very low exact‑match accuracy—1% for coding and 15.5% for sentencing—highlighting a gap between current benchmarks and the ability to reliably follow complex procedures.
By Utkarsh Soni, Syed Shariyar Murtaza, Yifan Nie, Sachin Chandrasekhar, Eugene Wen