Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation. We investigate whether verbalized confidence from black-box language models can serve as an actionable signal for retrieval routing.
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.
By Chandan Kumar Sah, Xiaoli Lian, Li Zhang
arXiv:2607. 07626v1 Announce Type: cross Abstract: Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability.
By Sahil Kale
The paper introduces matched trajectory replay, a protocol that fixes answer states, evidence points, budgets, and action costs to evaluate how confidence signals influence agent actions. Using this method, the authors compare raw verbalized confidence with post‑hoc isotonic calibration across six model‑dataset pairs, finding that calibration can significantly improve accuracy of committed answers but may reduce coverage and increase retrieval usage. The study concludes that calibration helps interpret commitment risk but does not predict the benefit of additional retrieval, indicating the need for separate value‑of‑information estimates.
By Prateek Chhikara
arXiv:2608. 07531v1 Announce Type: cross Abstract: Search-augmented language agents should retrieve external information only when necessary and ground their answers in retrieved evidence.
By Cheng Ruoxi, Ma Haoxuan, Zhang Hongyi, Zhang Junming, Duan Ranjie, Xia Qiaolin, Wang Hao, Lu Yu, Shi Haibo, Ma Xingjun
Search-G1 is a reward framework that encourages language agents to retrieve external information only when necessary and to ground their answers in that evidence. It uses two hidden‑state readouts—one predicting whether a closed‑book answer suffices and another measuring sensitivity to evidence deletion—to calibrate retrieval necessity and evidence reliance. The reward system gates evidence credit by necessity, rewards sufficient direct answers, and penalizes redundant searches, all without requiring costly annotations or LLM‑judge inference.
By Ruoxi Cheng, Haoxuan Ma, Hongyi Zhang, Junming Zhang, Ranjie Duan, Qiaolin Xia, Hao Wang, Yu Lu, Haibo Shi, Xingjun Ma