SciRerankBench is a benchmark designed to evaluate rerankers in retrieval‑augmented large language models (RAG‑LLMs) for scientific literature question answering. It covers five scientific subjects and introduces three types of question‑context‑answer pairs—Noisy Contexts, Semantically Similar but Logically Irrelevant Contexts, and Counterfactual Contexts—to test noise resilience, relevance disambiguation, and factual consistency. The study evaluates 13 popular rerankers across five LLM families, offering detailed insights into their strengths and limitations.
By Haotian Chen, Qingqing Long, Meng Xiao, Xiao Luo, Wei Ju, Chengrui Wang, Yuanchun Zhou, Xuezhi Wang, Hengshu Zhu
arXiv:2608. 03860v1 Announce Type: cross Abstract: We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19.
By Kaysarul Anas Apurba, Md. Hasibul Hasan, Rofiqul Alam Shehab, Asab Azad
Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that provide natural language access to scientific kno...
The paper presents a controlled comparison of six retrieval-augmented generation (RAG) strategies for scientific question answering on a large arXiv corpus. All pipelines use the same LLM generator and evaluation protocol, differing only in retrieval design—ranging from classic dense retrieval to late‑interaction methods like ColBERTv2. The authors also release a synthetic question dataset and code to enable reproducible, large‑scale evaluation of RAG trade‑offs.
By Bhagyesh Rathi, Eshan Chawla, William B. Andreopoulos
Lit3R is a system developed by tus-nlp for the LitTraceQA shared task, which focuses on evidence-grounded question answering over scientific literature. The system combines off-the-shelf retrieval, reranking, and large language model components without task-specific training, using an iterative retrieval process that merges BM25-based sparse and dense retrieval, cross-encoder reranking, and LLM verification, along with paper-to-paper expansion. In the official test set, Lit3R achieved a 4th place ranking on the leaderboard.
By Akira Ise, Kotaro Kumagai, Yuta Yamaguchi, Hisanori Ozaki, Yukio Uematsu, Ikuya Yamada
arXiv:2610.34391v1 Announce Type: cross
Abstract: Scientific literature in materials physics is growing faster than researchers can manually inspect it, yet most retrieval tools rely on keyword match...
By Ivan Iorsh, Aleksandr Kapitonov, Ivan Filianin, Matvey Lesonen, Nikita Chizhov
arXiv:2610.26951v1 Announce Type: new
Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...
By Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov
arXiv:2608.21252v1 Announce Type: cross
Abstract: Question answering (QA) over long, connected documents remains challenging because relevant evidence may span multiple entities and their relationshi...
By Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh, Jiawei Han
arXiv:2609.16519v1 Announce Type: new
Abstract: Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that...
By Bernie Boscoe, Srinath Saikrishnan, Vikram Seenivasan, Jack Stark, Andrew Lizarraga, Morgan Himes, Jonathan Soriano, PJ Allen, Tuan Do
arXiv:2610.27816v1 Announce Type: cross
Abstract: Retrieval-Augmented Generation (RAG) has become the dominant paradigm for grounding the outputs of large language models (LLM) in external knowledge....
By Niclas Hart, Nicolas Weeger
arXiv:2607. 24799v1 Announce Type: cross Abstract: Large Language Models tend to hallucinate when answering domain-specific ques tions from scientific documents without prior fine-tuning.
By Alexandru-Andrei Sauc\u{a}, Ana-Luiza Rusnac
The paper evaluates seven open‑source large language models for retrieval‑augmented generation in the ESG reporting domain, using 498 real‑world ESG reports from EU‑listed companies and 100 synthetic QA pairs. Performance is measured with RAGAS metrics, showing strong retrieval scores but variable generation quality, especially in faithfulness and factual correctness. The results highlight significant differences across model architectures and underscore the need for domain‑specific fine‑tuning to improve factual accuracy.
By Motaz Saad, Anna Borrelli, Ivan Gentile, Kianna Kazemi, Francesco Piccialli, Antonella Longo