The paper introduces XConf, an experiential confidence estimator that augments a language model’s current inference with a record of its past graded episodes. By recalling similar past tasks and reflecting on past outcomes, XConf generates confidence scores without accessing logits or updating weights, achieving superior discrimination and calibration across diverse benchmarks. The method demonstrates significant gains in selective prediction, improving success rates on agent tasks by up to 8.7 points.
arXiv:2508. 14390v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often express verbal confidence that is poorly aligned with actual correctness, limiting their reliability in safety-critical applications.
arXiv:2607. 07626v1 Announce Type: cross Abstract: Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability.
arXiv:2606. 03969v1 Announce Type: cross Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode.
arXiv:2607. 12397v1 Announce Type: new Abstract: LLM agents act in external environments where each action changes the state that later decisions condition on, and where a single wrong step can waste interaction budget or trigger irreversible side effects long before the final failure is observed.
arXiv:2606. 24281v1 Announce Type: cross Abstract: Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success.
LLM agents act in external environments where each action changes the state that later decisions condition on, and where a single wrong step can waste interaction budget or trigger irreversible side effects long before the final failure is observed. Reliable deployment therefore requires \emph{step-level confidence estimation}: a calibrated probability that each proposed action is productive, available \emph{before} the action is executed.
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.
arXiv:2509.24988v2 Announce Type: replace-cross Abstract: Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remain...
arXiv:2605. 27752v2 Announce Type: replace Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.
arXiv:2606. 32032v1 Announce Type: cross Abstract: Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one's own cognitive processes.
DirEAG introduces a Dirichlet Evidence Aggregation technique to calibrate verbalized confidence in large language models performing mathematical reasoning. By converting each elicited answer-confidence pair into calibrated soft evidence over candidate answers and a null state, it addresses prompt- and task-dependent bias that simple averaging or heuristic aggregation cannot handle. Experiments on GSM8K, SVAMP, and GSM-Hard with Qwen, Mistral, and Gemma models demonstrate that DirEAG achieves better calibration while maintaining competitive answer selection compared to existing methods.
arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...