Hugging Face Trending Papers

How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

The study presents an evidence‑aware evaluation protocol for a retrieval‑augmented generation system focused on breast lesion management literature. Using an 18‑document corpus and 18 queries, the authors compared dense embeddings, BM25, and cross‑encoder reranking across various chunk sizes and overlaps, finding that a hybrid retriever with 384‑token chunks and 20% overlap achieved top‑three evidence retrieval for 17 of 18 queries and high recall and precision metrics. Ground‑truth curation expanded evidence spans and accepted chunk targets, demonstrating that annotation coverage significantly affects retrieval performance.

arXiv Computation and Language
Oct 23

How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

The study introduces an evidence-aware evaluation protocol for a retrieval‑augmented generation system focused on breast lesion management literature. Using an 18‑document corpus and 18 queries, the authors compared dense embeddings, BM25, and cross‑encoder reranking across various chunk sizes and overlaps, finding that a hybrid retriever with 384‑token chunks and 20% overlap achieved top‑three retrieval success for 17 of 18 queries and high recall and precision metrics. Expanding ground‑truth relevance annotations increased recognized evidence spans and accepted chunk targets, though retrieval rankings remained unchanged, underscoring the sensitivity of evaluation to annotation coverage.

By Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias, Tha\'is Gaud\^encio, Yuri Malheiros, Barufaldi
Hugging Face Trending Papers
Jul 30

CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography

Large chest radiography archives are difficult to search because most studies are paired only with free-text reports rather than structured clinical annotations. Vision-language models offer a natural interface for text-to-image retrieval, but current biomedical models are primarily optimized for report-to-image matching rather than for satisfying short clinical search queries.

arXiv AI
Jul 20

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

arXiv:2604. 04593v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions.

By Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim, Tae-Joon Jeon
arXiv Computation and Language
Aug 28

Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG

The paper investigates whether incorporating an evidence-support signal into retrieval evaluation for retrieval‑augmented generation (RAG) improves downstream decision‑making. Across multiple benchmarks and a TREC RAG 2025 setting, the evidence signal alters retriever rankings but its benefits vary: it does not consistently enhance retriever training, its usefulness for system selection depends on generator instructions, and it does not reliably predict answer quality on unseen topics. Human filtering of evidence‑rich passages preserves useful content, yet evaluators disagree on whether this improves final answers, indicating that evidence‑aware evaluation alone does not guarantee better downstream outcomes.

By Utshab Kumar Ghosh, Debayan Mukhopadhyay, Shubham Chatterjee
arXiv Computation and Language
Sep 28

Where Does Retrieval-Based Open-Ended Evaluation Fail? Automatic Taxonomy Induction from Long-Form Medical Answer Factuality Verification

The paper investigates why retrieval‑based open‑ended evaluation fails in medical fact verification. By creating two detailed taxonomies—one for retrieval‑stage errors across five quality dimensions and another for verifier‑reasoning errors across six steps—the authors automatically label evidence quality and reasoning errors using an LLM‑as‑Judge pipeline. Their large‑scale stress tests across multiple retrieval methods and verifier models show that increasing model size, reasoning effort, source breadth, or medical fine‑tuning does not eliminate these failure modes, indicating fundamental limits of the retrieve‑then‑verify paradigm in open‑ended medical contexts.

By Heyuan Huang, Jirui Dai, Alexandra DeLucia, Sonal Joshi, Mahsa Yarmohammadi, Jie Gao, Bernal Jim\'enez Guti\'errez, Mark Dredze
arXiv AI
Jul 9

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

arXiv:2607. 06641v1 Announce Type: cross Abstract: Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hallucinations and the rapid evolution of official guidance.

By Felix Feldman, Joshua Harris, Timothy Laurence, Leo Loman, Ollie Higgins, Fan Grayson, Poonam Soma, Bethany Pace-Bonello, Michael Borowitz, Toby Nonnenmacher
arXiv AI
Jul 13

Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs

arXiv:2508. 14817v2 Announce Type: replace-cross Abstract: Objective: To evaluate whether retrieval-augmented generation (RAG) can serve as an efficient alternative to long-context prompting for clinical reasoning over electronic health records (EHRs).

By Skatje Myers, Dmitriy Dligach, Timothy A. Miller, Samantha Barr, James Landefeld, Yanjun Gao, Matthew Churpek, Anoop Mayampurath, Majid Afshar