arXiv:2603. 24929v2 Announce Type: replace Abstract: Understanding and quantifying uncertainty in large language model (LLM) outputs is critical for reliable deployment.
By Farhan Ahmed, Yuya Jeremy Ong, Chad DeLuca
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
The paper challenges the common practice of estimating aleatoric uncertainty in large language models (LLMs) by generating multiple clarified inputs and comparing the resulting answers. It argues that answers are unnecessary, costly, and can introduce epistemic leakage, proposing instead a clarification-only method that directly assesses ambiguity from the space of plausible interpretations. Experiments on three benchmarks show the new approach improves AUROC, reduces computational cost, and yields uncertainty estimates less correlated with epistemic uncertainty.
By Omer Nahum, Niv Nayman, Jonathan Fhima, Alon Zolfi, Jeremy Levy, Shai Mazor, Paolo Favaro
Intent Engine is a natural‑language intent translation architecture that converts user intents into validated Service‑level Objectives (SLOs) for compute‑continuum microservice placement. It combines schema‑constrained extraction, retrieval‑grounded value construction from monitored infrastructure, and validation against supported constraints to produce reliable SLO artifacts. In evaluations on a 716‑record dataset, Intent Engine outperformed prompting baselines and a rule‑based parser, achieving a 0.941 total F1 score with GPT‑4.1 mini and reducing downstream placement failures from 30.8% to 2.1%.
By Koushikur Islam, Rodrigo N. Calheiros
The paper introduces a perturbation-based framework for detecting input ambiguity in large language models (LLMs) before attempting to disambiguate. It presents two detectors—a black-box rule-based perturbation library and a white-box importance-guided method that leverages internal model signals—both designed to preserve the original input’s label while remaining close to it. Experiments on open-domain, conversational, and visual QA datasets show these methods outperform existing output-side uncertainty and LLM-paraphrase-based baselines, underscoring the value of early ambiguity detection.
By Tiejin Chen, Xiaoou Liu, Michael Clifford, Ahmadreza Moradipari, Yitao Chen, Kyungtae Han, Nejib Ammar, Hua Wei
The study investigates whether lightweight open LLMs can reliably translate operator-stated intents into enforceable service-level objectives (SLOs) for intent-based networking (IBN). Ten LLMs with fewer than 10 B parameters were evaluated across five dimensions—translation accuracy, intent validation, hallucination, energy usage, and robustness to phrasing—using 716 placement intents. The best configurations achieved a strict F1 of 0.943, rejected 191 of 195 invalid intents, and incurred 0.044–0.148 Wh of GPU energy per translation, while realistic rephrasing reduced invalid intent rejection in most configurations.
By Sabbir Ahmed, Hazem Abbas, Salimur Choudhury