arXiv AI August 13, 2026 By Jeonghwan Choi, Taewon Yun, Minjeong Ban, Gyeonghun Sun, Jae-Gil Lee, Hwanjun Song

Towards Query-Agnostic RAG Evaluation via Query Coverage and Claim Verifiability

arXiv:2608. 11238v1 Announce Type: new Abstract: Retrieval-augmented generation improves the factuality of large language models by grounding responses in retrieved evidence, yet existing evaluation frameworks struggle to provide consistent, fine-grained diagnostics across the diverse spectrum of user queries, ranging from close-ended fact-seeking to open-ended explanatory requests.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 9

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

arXiv:2607. 06641v1 Announce Type: cross Abstract: Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hallucinations and the rapid evolution of official guidance.

By Felix Feldman, Joshua Harris, Timothy Laurence, Leo Loman, Ollie Higgins, Fan Grayson, Poonam Soma, Bethany Pace-Bonello, Michael Borowitz, Toby Nonnenmacher

llms rag nlp benchmarks safety

arXiv AI

2d ago

Fine-grained Claim-level RAG Benchmark for Law

arXiv:2605. 21071v4 Announce Type: replace-cross Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs generate responses.

By Souvick Das, Sallam Abualhaija, Domenico Bianculli

llms rag benchmarks safety

arXiv AI

Jun 4

QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuples

arXiv:2606. 04646v1 Announce Type: cross Abstract: Many real-world questions over business, legal, and scientific corpora are natural-language versions of database-style queries over records latent in text.

By Mengao Zhang, Xiang Yang, Chang Liu, Tianhui Tan, Ke-wei Huang

llms rag nlp benchmarks data-engineering

arXiv AI

Jun 16

EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries

arXiv:2606. 15735v1 Announce Type: cross Abstract: Discharge summaries are crucial clinical documents containing the context of a patient's overall hospital stay, and are routinely reviewed by medical experts for patient readmission, ongoing care, and diagnostic decision-making.

By Jiyoun Kim, Muhan Yeo, Eunhye Jang, Jeewon Yang, Hangyul Yoon, Su Ji Lee, Hee Jo Han, Hee-Jae Jung, Doyun Kwon, Jun young Lee, Jaehun Lee, Jung-Oh Lee, Sunjun Kweon, Jong Hak Moon, Daseul Kim, Minjae Cho, Edward Choi

llms nlp benchmarks

arXiv AI

Jul 20

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

arXiv:2604. 04593v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions.

By Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim, Tae-Joon Jeon

llms rag nlp benchmarks

arXiv AI

Jul 1

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA

arXiv:2504. 07385v3 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) become increasingly used for question-answering (QA), relying on static, pre-annotated references for evaluation poses significant challenges in cost, scalability, and completeness.

By Sher Badshah, Ali Emami, Hassan Sajjad

llms agents benchmarks

Related stories

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

Fine-grained Claim-level RAG Benchmark for Law

QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuples

EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA