The paper introduces a perturbation-based framework for detecting input ambiguity in large language models (LLMs) before attempting to disambiguate. It presents two detectors—a black-box rule-based perturbation library and a white-box importance-guided method that leverages internal model signals—both designed to preserve the original input’s label while remaining close to it. Experiments on open-domain, conversational, and visual QA datasets show these methods outperform existing output-side uncertainty and LLM-paraphrase-based baselines, underscoring the value of early ambiguity detection.
Large language models (LLMs) often answer ambiguous inputs as if they had a single intended meaning. This is problematic because input ambiguity is an upstream source of uncertainty: when a prompt adm...
The paper challenges the common practice of estimating aleatoric uncertainty in large language models (LLMs) by generating multiple clarified inputs and comparing the resulting answers. It argues that answers are unnecessary, costly, and can introduce epistemic leakage, proposing instead a clarification-only method that directly assesses ambiguity from the space of plausible interpretations. Experiments on three benchmarks show the new approach improves AUROC, reduces computational cost, and yields uncertainty estimates less correlated with epistemic uncertainty.
VIDA (Visually-Dependent Ambiguity) is a new dataset comprising 2,500 curated instances designed to test whether multimodal machine translation models can use visual input to resolve ambiguous source spans. The authors also introduce Disambiguation-Centric Metrics, which employ an LLM-as-a-judge classifier to verify correct span-level resolution. Experiments with recent large vision‑language models show that visual disambiguation remains difficult, and that chain‑of‑thought supervised fine‑tuning yields stronger out‑of‑distribution performance, especially for collective‑noun ambiguities.
arXiv:2609.37594v1 Announce Type: new Abstract: Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot t...
arXiv:2610.20052v1 Announce Type: new Abstract: Ambiguous inputs can admit multiple interpretations, each potentially yielding a different answer. Existing work largely treats this ambiguity as a pro...
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv:2606. 19868v1 Announce Type: new Abstract: Although large language models (LLMs) have shown strong capabilities across a wide range of tasks, their outputs often remain unreliable and may contain hallucinations, making uncertainty estimation (UE) essential for building trustworthy LLMs.
arXiv:2606. 02093v1 Announce Type: cross Abstract: The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ).
Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors.
arXiv:2606. 05486v1 Announce Type: cross Abstract: Prompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens.
The paper introduces Causal-Invariant Masking (CIM) to better quantify epistemic uncertainty in Multimodal Large Language Models (MLLMs) by measuring semantic shift between original predictions and those conditioned on a causally-focused view. It proposes Semantic Divergence as a core metric that converges to the variance of the model’s sensitivity to non‑causal correlations, and introduces Expected Embedding Drift (EED) as a fast geometric proxy that estimates this shift directly in the embedding space. Experiments demonstrate state‑of‑the‑art uncertainty quantification performance and a nearly 50% speedup with EED.
arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.