arXiv:2607. 10738v1 Announce Type: cross Abstract: Recent advances in equipping Large Language Models (LLMs) with search tools and outcome-reward reinforcement learning (RL) have achieved new state-of-the-art results on open-domain QA tasks.
Recent advances in equipping Large Language Models (LLMs) with search tools and outcome-reward reinforcement learning (RL) have achieved new state-of-the-art results on open-domain QA tasks. However, we argue that current training paradigms harbor a critical vulnerability: they predominantly reward correct answers but fail to penalize fabricated ones when retrieval fails, thereby implicitly exacerbating hallucinations.
The paper investigates whether language models can learn to abstain from answering when uncertain, focusing on Qwen3-14B for factual questions. Two approaches are examined: training the model itself and training a separate answer selector. The learned selector improves utility over the model’s calibrated confidence filter on TriviaQA, raising coverage from 20.2% to 30.9% and reducing error rate from 23.0% to 14.5%, though gains are inconsistent across datasets and reinforcement‑learning changes do not consistently help.
The paper introduces matched trajectory replay, a protocol that fixes answer states, evidence points, budgets, and action costs to evaluate how confidence signals influence agent actions. Using this method, the authors compare raw verbalized confidence with post‑hoc isotonic calibration across six model‑dataset pairs, finding that calibration can significantly improve accuracy of committed answers but may reduce coverage and increase retrieval usage. The study concludes that calibration helps interpret commitment risk but does not predict the benefit of additional retrieval, indicating the need for separate value‑of‑information estimates.
arXiv:2509. 25760v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have demonstrated strong performance on factoid question answering, they are still prone to hallucination and untruthful responses, particularly when tasks demand information outside their parametric knowledge.
The paper introduces RAG‑Stress, a diagnostic protocol that tests how retrieval‑augmented generation models handle misleading evidence. By fixing the question and reference answer, editing a single assertion to support an incorrect answer, and varying source‑priority policies and evidence positions, the study measures misleading rate (MR) and clean accuracy across fifteen systems on multiple QA datasets. Results show that prioritizing documents increases MR by 10.9–13.5 percentage points, with the highest MR when the misleading span appears at the end of the evidence.
arXiv:2604. 03904v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often produce confident but incorrect answers, in part because standard evaluation incentives reward guessing over expressing uncertainty.
arXiv:2511.11500v3 Announce Type: replace Abstract: Modern language models fail a fundamental requirement of trustworthy intelligence: knowing when not to answer. Despite achieving impressive accurac...
arXiv:2610.35059v1 Announce Type: cross Abstract: Large language models (LLMs) can produce plausible but incorrect answers when they are uncertain, which can mislead users and undermine model reliabi...
arXiv:2607. 01612v1 Announce Type: new Abstract: Training large language models (LLMs) with reinforcement learning (RL) has significantly advanced their performance on reasoning and question-answering tasks.
arXiv:2507.21931v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....
arXiv:2607. 04332v1 Announce Type: new Abstract: In this paper, we consider the setting where large language models (LLMs) are trained using reinforcement learning (RL) to simultaneously improve reasoning accuracy and verbalize its confidence.