Clarify-Then-Search is a benchmark that tests whether large language models can ask clarification questions to improve the usefulness of deep search results. It uses 518 real-world query pairs from Baidu, where each intent query is paired with an underspecified version. The evaluation involves a clarifier asking up to three questions, a user answerer providing only explicit information, and a rewriter generating a new query that is then searched; performance is measured by a weighted nugget-recall score.
By Deqiang Huang, Jingbo Zhou, Xinjiang Lu, Tong Xu, Hua Wu, Enhong Chen
arXiv:2609.07093v2 Announce Type: replace
Abstract: Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely a...
By Yifan Wang, Xinkui Lin, Yongxiu Xu, Shen Gao, Ruochen Yang, Kun Huang, Yubin Wang, Jie Wu, Wei Liu, Jian Luan, Hongbo Xu, Shuo Shang
The paper introduces LOCOMO-CONV, a conversational memory benchmark that expands on the existing LoCoMo dataset with four query styles—dialog, implicit, counterfactual, and composed—designed to evaluate memory systems in realistic conversational settings. Experiments across five memory systems reveal that conversational framing uncovers significant retrieval gaps missed by traditional QA benchmarks, particularly for implicit and composed queries, and that strong retrieval does not necessarily translate into higher response quality. The study also highlights silent grounding in implicit queries, where memory enhances contextual grounding without explicitly presenting the gold fact, suggesting a need for reasoning-based memory elaboration.
By Wen-Yu Chang, Yun-Nung Chen
ConfRAG introduces a confidence-guided approach to reduce hallucinations in large language models and selectively trigger Retrieval-Augmented Generation (RAG) only when the model is uncertain. The ConfQA fine‑tuning strategy trains the model to answer correctly or respond with "I am unsure," achieving a drop in hallucination rates from 20‑40% to below 5% across factuality benchmarks. Building on ConfQA, ConfRAG limits external retrievals by more than 30% while maintaining over 95% accuracy in ideal scenarios.
By Yin Huang, Yifan Ethan Xu, Kai Sun, Vera Yan, Alicia Sun, Haidar Khan, Jimmy Nguyen, Jingxiang Chen, Mohammad Kachuee, Zhaojiang Lin, Yue Liu, Aaron Colak, Anuj Kumar, Wen-tau Yih, Xin Luna Dong
arXiv:2606. 04646v1 Announce Type: cross Abstract: Many real-world questions over business, legal, and scientific corpora are natural-language versions of database-style queries over records latent in text.
By Mengao Zhang, Xiang Yang, Chang Liu, Tianhui Tan, Ke-wei Huang
arXiv:2609.37469v1 Announce Type: cross
Abstract: Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without...
By Suting Chen, Peichun Hua, Yunming Xiao
arXiv:2610.20162v1 Announce Type: cross
Abstract: Language models often produce semantically similar answers to open-ended questions, even when many plausible alternatives exist. Generating useful al...
By Xucheng Yu, Haibo Jin, Xiaoqi Wang, Runze Kong, Haohan Wang
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.
By Chandan Kumar Sah, Xiaoli Lian, Li Zhang
The paper introduces a method for determining whether retrieval-augmented generation (RAG) systems have sufficient, insufficient, or conflicting evidence to answer a question. By training a lightweight linear classifier on hidden activations and attention-derived features from 16 language models, the authors demonstrate that these internal signals reliably predict the adequacy of retrieved documents, outperforming prompting-based baselines and specialized RAG models. Analysis shows that middle-layer hidden states carry the most informative signals for this triage task.
By Syed Mahbubul Huq, Christopher Child, Tillman Weyde, Pranava Madhyastha
The paper evaluates how robust large language models (LLMs) are when using retrieval‑augmented generation (RAG) in practical settings. It investigates whether RAG always outperforms non‑RAG approaches, whether adding more retrieved documents helps, and whether the order of documents matters, using a benchmark of 1,891 samples across five datasets and three task categories. Experiments with 11 LLMs show generally high retrieval robustness, but performance varies by task and prompting strategy, indicating that adopting RAG should be considered on a case‑by‑case basis.
By Shuyang Cao, Karthik Radhakrishnan, David Rosenberg, Steven Lu, Pengxiang Cheng, Lu Wang, Shiyue Zhang
arXiv:2609.24372v1 Announce Type: new
Abstract: In-context learning (ICL) based on large language models (LLMs) has shown promising potential in alleviating performance bottlenecks caused by the limi...
By Jingyu Wang, Shijie Wu, Fusheng Jin
The paper introduces EAR, an Entity‑Aware Partitioning approach that improves retrieval‑augmented generation for multiple‑choice question answering by extracting normalized surface anchors from questions, answers, and the corpus. EAR retrieves local windows around matching anchors and can attach a larger parent passage via an extractive summary, reducing retrieved words by 37.5‑40.2% compared to fixed‑size chunks. Experiments on a cleaned MMLU‑style subset with Mistral, Gemma, and DeepSeek show modest accuracy changes, none statistically significant, highlighting EAR’s methodological contribution of compact, inspectable retrieval units.
By Cenab Batu Bora, Oylum Alatl{\i}, Sebnem Bora, Oguz Dikenelli