The paper introduces a perturbation-based framework for detecting input ambiguity in large language models (LLMs) before attempting to disambiguate. It presents two detectors—a black-box rule-based perturbation library and a white-box importance-guided method that leverages internal model signals—both designed to preserve the original input’s label while remaining close to it. Experiments on open-domain, conversational, and visual QA datasets show these methods outperform existing output-side uncertainty and LLM-paraphrase-based baselines, underscoring the value of early ambiguity detection.
By Tiejin Chen, Xiaoou Liu, Michael Clifford, Ahmadreza Moradipari, Yitao Chen, Kyungtae Han, Nejib Ammar, Hua Wei
The paper challenges the common practice of estimating aleatoric uncertainty in large language models (LLMs) by generating multiple clarified inputs and comparing the resulting answers. It argues that answers are unnecessary, costly, and can introduce epistemic leakage, proposing instead a clarification-only method that directly assesses ambiguity from the space of plausible interpretations. Experiments on three benchmarks show the new approach improves AUROC, reduces computational cost, and yields uncertainty estimates less correlated with epistemic uncertainty.
By Omer Nahum, Niv Nayman, Jonathan Fhima, Alon Zolfi, Jeremy Levy, Shai Mazor, Paolo Favaro
VIDA (Visually-Dependent Ambiguity) is a new dataset comprising 2,500 curated instances designed to test whether multimodal machine translation models can use visual input to resolve ambiguous source spans. The authors also introduce Disambiguation-Centric Metrics, which employ an LLM-as-a-judge classifier to verify correct span-level resolution. Experiments with recent large vision‑language models show that visual disambiguation remains difficult, and that chain‑of‑thought supervised fine‑tuning yields stronger out‑of‑distribution performance, especially for collective‑noun ambiguities.
By Jingheng Pan, Xintong Wang, Longyue Wang, Liang Ding, Weihua Luo, Chris Biemann
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors.
arXiv:2606. 19868v1 Announce Type: new Abstract: Although large language models (LLMs) have shown strong capabilities across a wide range of tasks, their outputs often remain unreliable and may contain hallucinations, making uncertainty estimation (UE) essential for building trustworthy LLMs.
By Jiayi Wang, Xu-Yao Zhang
arXiv:2610.20052v1 Announce Type: new
Abstract: Ambiguous inputs can admit multiple interpretations, each potentially yielding a different answer. Existing work largely treats this ambiguity as a pro...
By Jingqin Tang, Tim Tomov, Sebastian Schmidt, Dominik Fuchsgruber, Stephan G\"unnemann
arXiv:2609.37594v1 Announce Type: new
Abstract: Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot t...
By David Achara, Maryam Sultana, Alexander D. Rast, Fabio Cuzzolin
arXiv:2603. 24967v2 Announce Type: replace Abstract: Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment.
By Aditya Taparia, Ransalu Senanayake, Kowshik Thopalli, Vivek Narayanaswamy
arXiv:2606. 02093v1 Announce Type: cross Abstract: The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ).
By Ieva Raminta Stali\=unait\.e, James Bishop, Andreas Vlachos
arXiv:2606. 05486v1 Announce Type: cross Abstract: Prompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens.
By Govind Ramesh, Yao Dou, Wei Xu
arXiv:2607.05721v2 Announce Type: replace
Abstract: Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refin...
By Yimeng Zhang, Yingying Zhuang, Ziyi Wang, Yuxuan Lu, Pei Chen, Aman Gupta, Zhe Su, Ming Tan, Zhilin Zhang, Qun Liu, Manikandarajan Ramanathan, Rajashekar Maragoud, Edward Vul, Jing Huang, Dakuo Wang