arXiv AI
Reasoning-Aware Multicalibrated Reliability Predictors for Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 11
Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
arXiv:2606. 11211v1 Announce Type: cross Abstract: The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment.
arXiv AI
Jun 3
Quantifying Faithful Confidence Expression in Large Reasoning Models
arXiv:2606. 03969v1 Announce Type: cross Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode.
arXiv AI
Aug 5
The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
arXiv:2608. 03291v1 Announce Type: cross Abstract: Chain-of-thought (CoT) reasoning improves large language model (LLM) performance while also providing an observable interface to the model's reasoning process.
arXiv Machine Learning
Jun 2
FLaG: Fine-Grained Latent Grouping for Hallucination Detection
arXiv:2606. 00301v1 Announce Type: new Abstract: Hallucinations in large language models (LLMs) arise from heterogeneous failure mechanisms, making reliable detection difficult for any single global uncertainty score.
arXiv Computation and Language
Oct 16
Reasoning Error from Known Fact: Step-Level Self-Consistency Group Relative Policy Optimization for LLMs
The paper introduces Step-level Self-Consistency Group Relative Policy Optimization (SSC‑GRPO), a method that assigns rewards to individual steps in a reasoning trace by evaluating self‑consistency across multiple rollouts. It targets Context‑Sensitive Factual Hallucinations—errors that arise when a model has the correct knowledge but produces false statements due to contextual interference. SSC‑GRPO outperforms previous approaches on mathematical reasoning benchmarks and hallucination leaderboards, offering a new way to detect and reduce hallucinations in large language model reasoning.
arXiv Computation and Language
Aug 25
Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models
arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...