arXiv AI
Aug 13

A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench

arXiv:2608. 12138v1 Announce Type: cross Abstract: General-purpose large language models (LLMs) have recently been reported to match or exceed specialized clinical AI tools on medical benchmarks, but such comparisons draw on a narrow set of systems and on benchmarks developed largely in high-income settings.

By Praveen Reddy, Charuta Mandke, Suvrankar Datta, Sarah Khan, Siddharth Reddy Anthireddy, Shitij Arora, Vishal Singh
arXiv AI
2d ago

Benchmarking Large Language Models on Heterogeneous Cognitive and Biomedical Knowledge Graphs: The Role of Graph Grounding in Multi-Hop Reasoning and Hallucination Mitigation

The paper benchmarks five large language models—GPT‑5.1, Gemini 2.0 Pro, Llama 3, Qwen, and BioGPT—across five heterogeneous knowledge graphs in biomedical and cognitive neuroscience. Using a multitier query dataset, it evaluates multi‑hop reasoning and factual accuracy under graph‑grounded and graph‑agnostic conditions, finding that general‑purpose models outperform domain‑specific ones on complex reasoning tasks and that graph grounding consistently reduces hallucinations. The study offers practical guidance for selecting LLM‑KG integration strategies and discusses hybrid neurosymbolic architectures for applications such as personalized medicine and explainable AI.

By Ali Sarabadani, Kheirolah Rahsepar Fard, Tahereh Jeirani