The paper challenges the common practice of estimating aleatoric uncertainty in large language models (LLMs) by generating multiple clarified inputs and comparing the resulting answers. It argues that answers are unnecessary, costly, and can introduce epistemic leakage, proposing instead a clarification-only method that directly assesses ambiguity from the space of plausible interpretations. Experiments on three benchmarks show the new approach improves AUROC, reduces computational cost, and yields uncertainty estimates less correlated with epistemic uncertainty.
By Omer Nahum, Niv Nayman, Jonathan Fhima, Alon Zolfi, Jeremy Levy, Shai Mazor, Paolo Favaro
The paper introduces ClariEdge, a dataset of about 3,900 examples that capture ambiguous or ill‑formed user queries in retrieval‑augmented generation contexts, including negative cases that do not need clarification. A lightweight LoRA fine‑tuned model achieves 96.6% classification accuracy, outperforming larger prompted baselines in both classification and clarification generation. The study highlights the importance of hard negatives for distinguishing truly ambiguous queries and demonstrates that the same model can be applied before and after retrieval in a RAG pipeline.
By Lihong He, Lucian Popa, Marina Danilevsky
arXiv:2606. 02093v1 Announce Type: cross Abstract: The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ).
By Ieva Raminta Stali\=unait\.e, James Bishop, Andreas Vlachos
arXiv:2609.24002v1 Announce Type: new
Abstract: Large language model agents increasingly answer financial questions by searching regulatory filings. Such questions are often deceptively under-specifi...
By Xinyu Wang, Tung Sum Thomas Kwok, Zhenghan Tai, Guang Cheng
arXiv:2607. 22554v1 Announce Type: new Abstract: Large language models (LLMs) often achieve strong accuracy on benchmarks, yet it remains unclear how reliably they apply this knowledge when the same question is phrased in different but equivalent ways.
By Kazem Faghih, Yize Cheng, Shoumik Saha, Mobina Pournemat, Armin Gerami, Soheil Feizi
arXiv:2610.07894v1 Announce Type: cross
Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is in...
By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Borui Jiang, Bo Han
arXiv:2608. 11631v1 Announce Type: new Abstract: In open-domain human-computer interaction scenarios, large language models (LLMs) frequently encounter user queries that are ambiguous or incomplete.
By Kuangzhao Yang, Ziliang Zhao, Zhicheng Dou
arXiv:2606. 26101v1 Announce Type: cross Abstract: Reliable evaluation of large language models should separate supported answering from unsupported guessing without conflating either with data contamination, prompt idiosyncrasy, or generic refusal behavior.
By Renwei Meng, Bowen Zhang, Jian Wang, Xican Wang, Haoyi Wu, Xuanyan Qiu, Shengan Yang
arXiv:2608. 03177v1 Announce Type: new Abstract: How can question answering (QA) systems determine whether a query is ambiguous?
By Jiwon Lee, Yong-chan Park, Jungin Hong, U Kang
Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust an individual answer...
arXiv:2605.25831v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) define a distribution over text, which can be viewed as a probabilistic representation of uncertainty: sampling...
By Joris Baan, Wilker Aziz, Barbara Plank, Raquel Fern\'andez
arXiv:2511. 10453v4 Announce Type: replace-cross Abstract: Large language models often respond to ambiguous requests by implicitly committing to one interpretation, frustrating users and creating safety risks when that interpretation is wrong.
By Irina Saparina, Mirella Lapata