arXiv:2607. 16868v1 Announce Type: new Abstract: Large Language Models (LLMs) often produce confidently stated yet unreliable outputs, posing critical challenges for deployment in safety-sensitive applications.
arXiv:2607. 17266v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing.
arXiv:2607. 27405v2 Announce Type: replace-cross Abstract: Both expressions of uncertainty and inferences are ubiquitous in natural language, and valid inferences over natural-language expressions of uncertainty are necessary for not only everyday conversations but also for high-stakes domains such as medicine and law.
arXiv:2511. 16275v4 Announce Type: replace-cross Abstract: Reliable uncertainty quantification (UQ) is essential for deploying large language models (LLMs) in safety-critical scenarios, as it enables them to abstain from responding when uncertain, thereby avoiding hallucinations, i.
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv:2606. 09800v1 Announce Type: cross Abstract: Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle.
The paper introduces GUT, a method that uses directed acyclic graphs to represent all possible reasoning branches of Large Language Models (LLMs). It comprises two modules: GUT-Q, which quantifies reasoning uncertainty by approximating graph complexity, and GUT-O, which reduces uncertainty through reinforcement learning that rewards lower uncertainty. Experiments on four LLMs across five datasets demonstrate GUT’s effectiveness in measuring and mitigating reasoning uncertainty.
arXiv:2607. 08017v1 Announce Type: cross Abstract: Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps.
arXiv:2603. 24967v2 Announce Type: replace Abstract: Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment.
The paper presents a geometric framework for quantifying uncertainty in large language models (LLMs) at both the prompt and answer levels. By modeling a prompt-conditioned semantic distribution in answer embedding space and using archetypal analysis on multiple sampled answers, the method estimates distribution entropy for prompt-level uncertainty and atypicality for individual answer reliability. Experiments demonstrate comparable or superior performance to existing techniques on short-form QA datasets and notably better results on medical datasets where hallucinations pose critical risks.
Semantic Self-Distillation (SSD) is a method that distills the semantic dispersion of sampled answers from large language models into lightweight student models. These students estimate a prompt-conditioned density before answer generation, providing a prompt-level uncertainty signal via entropy and an answer-level reliability measure through probability density. Experiments on TriviaQA and MMLU show that SSD matches the teacher’s uncertainty estimates while enabling additional tasks such as hallucination prediction, out-of-domain detection, and multiple-choice answer selection.
arXiv:2606. 02093v1 Announce Type: cross Abstract: The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ).
The paper introduces Divergent Token Confidence (DTC), a method that estimates large language model confidence by counting tokens where two models strongly disagree during decoding. DTC uses Jensen-Shannon divergence between next-token distributions along the same reasoning trajectory and shows a near-negative correlation with answer accuracy. Experiments on multiple model families and six mathematical benchmarks demonstrate that DTC improves calibration over traditional probability-based and verbalized baselines, achieving lower expected calibration errors in both white-box and black-box settings.