arXiv AI By Brendan King, Farima Fatahi Bayat, Jean-Flavien Bussotti, Pouya Pezeshkpour, Estevam Hruschka

Confidence Reasoning Graphs: Structured Confidence Estimation for LLM Agents

Read the original on arXiv AI →

The paper introduces Confidence Reasoning Graphs (CRGs), an inference-time framework that estimates the probability an LLM agent has successfully completed its task using a single trajectory without requiring privileged model access or training data. CRGs decompose the agent’s overall claim of success into contextualized sub-claims, estimate confidence for each terminal claim based on trajectory evidence, and aggregate these into an overall confidence estimate. Experiments across multiple benchmarks, models, and agent frameworks show that CRGs produce better-calibrated confidence and more effective risk-aware decision making than existing verbalized, sampling-based, and white-box surrogate methods, while also providing transparent, auditable evidence for each estimate.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Sep 22

Monitorable Chart Reasoning Agents via Verifiable Process Rewards

The paper introduces Chart‑RVR, a reinforcement‑learning framework that trains chart‑reasoning agents to produce monitorable, verifiable outputs. It decomposes reasoning into three auditable stages—Structure, Evidence, and Derivation—allowing stakeholders to trace how the model reads the chart, extracts data, and computes the answer. Experiments on six benchmarks show that Chart‑RVR matches or exceeds state‑of‑the‑art accuracy while delivering rationales that are far more verifiable and evidence‑grounded than existing methods.

By Sanchit Sinha, Oana Frunza, Kashif Rasul, Aidong Zhang
arXiv AI
Sep 17

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents

The paper introduces XConf, an experiential confidence estimator that augments a language model’s current inference with a record of its past graded episodes. By recalling similar past tasks and reflecting on past outcomes, XConf generates confidence scores without accessing logits or updating weights, achieving superior discrimination and calibration across diverse benchmarks. The method demonstrates significant gains in selective prediction, improving success rates on agent tasks by up to 8.7 points.

By Caiqi Zhang, Xiaochen Zhu, Chengzu Li, Yulong Chen, Dharshan Kumaran, Nigel Collier
arXiv Computation and Language
Aug 27

ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability

ReFIne is a training framework that augments large reasoning models with three trustworthiness properties: interpretability, faithfulness, and reliability. It combines supervised fine‑tuning with GRPO to produce structured, tag‑based reasoning traces, explicitly disclose decisive information, and provide self‑assessments of soundness and confidence. Applied to Qwen3 models, ReFIne improves interpretability by 44.0 %, faithfulness by 18.8 %, and reliability by 42.4 % on mathematical benchmarks.

By Chung-En Sun, Ge Yan, Akshay Kulkarni, Tsui-Wei Weng
Hugging Face Trending Papers
Jul 14

Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agents

LLM agents act in external environments where each action changes the state that later decisions condition on, and where a single wrong step can waste interaction budget or trigger irreversible side effects long before the final failure is observed. Reliable deployment therefore requires \emph{step-level confidence estimation}: a calibrated probability that each proposed action is productive, available \emph{before} the action is executed.

arXiv AI
Jul 15

Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agents

arXiv:2607. 12397v1 Announce Type: new Abstract: LLM agents act in external environments where each action changes the state that later decisions condition on, and where a single wrong step can waste interaction budget or trigger irreversible side effects long before the final failure is observed.

By Yaopei Zeng, Congchao Wang, JianHang Chen, Nan Wang, Yurui Chang, Lu Lin