arXiv AI
AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Sep 15
AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research
Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that provide natural language access to scientific kno...
arXiv AI
Jul 24
SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration
arXiv:2607. 20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources.
arXiv AI
Oct 15
SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs
SciRerankBench is a benchmark designed to evaluate rerankers in retrieval‑augmented large language models (RAG‑LLMs) for scientific literature question answering. It covers five scientific subjects and introduces three types of question‑context‑answer pairs—Noisy Contexts, Semantically Similar but Logically Irrelevant Contexts, and Counterfactual Contexts—to test noise resilience, relevance disambiguation, and factual consistency. The study evaluates 13 popular rerankers across five LLM families, offering detailed insights into their strengths and limitations.
arXiv AI
Jul 2
GRACE-RAG: Governed Retrieval Architecture for Canonical Evidence Synthesis, Enabling Lightweight Deployment in Closed-Domain Institutional Settings
arXiv:2607. 00013v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are widely used in institutional question answering settings where responses must be grounded in authoritative documentation (Gao et al.
arXiv AI
Aug 12
InfiniteScienceGym: An Unbounded, Procedurally-Generated Benchmark for Scientific Analysis
arXiv:2604. 13201v2 Announce Type: replace-cross Abstract: Large language models are emerging as scientific assistants, but evaluating their ability to reason from empirical data remains challenging.
arXiv AI
Oct 1
Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering
The paper presents a controlled comparison of six retrieval-augmented generation (RAG) strategies for scientific question answering on a large arXiv corpus. All pipelines use the same LLM generator and evaluation protocol, differing only in retrieval design—ranging from classic dense retrieval to late‑interaction methods like ColBERTv2. The authors also release a synthetic question dataset and code to enable reproducible, large‑scale evaluation of RAG trade‑offs.