arXiv Machine Learning June 5, 2026 By Mohammad Anas Jawad, Cornelia Caragea

CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction

Read the original on arXiv Machine Learning →

arXiv:2606. 05799v1 Announce Type: new Abstract: Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's {\em behavioral robustness} to irrelevant or misleading information.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 24

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.

By Matthew ffrench-Constant, Daniel Yang, Xinmeng Huang, Sanyam Kapoor

llms benchmarks

arXiv AI

Aug 6

Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models

arXiv:2608. 05064v1 Announce Type: cross Abstract: Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human.

By Jianru Shen

llms

arXiv AI

Jun 9

The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust

arXiv:2606. 07822v1 Announce Type: cross Abstract: As language models improve and become increasingly deployed to solve a variety of tasks, trustworthiness becomes essential.

By Nishant Subramani, Palash Goyal, Yiwen Song, Mani Malek, Yuan Xue, Tomas Pfister, Hamid Palangi

llms nlp

arXiv AI

Aug 17

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

arXiv:2608. 13591v1 Announce Type: new Abstract: High-confidence errors in large language models are often treated as evidence of fragile internal inference.

By Akira Okutomi

llms

arXiv AI

Jun 11

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

arXiv:2606. 11211v1 Announce Type: cross Abstract: The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment.

By Prakul Sunil Hiremath, Harshit R. Hiremath

llms

arXiv Computation and Language

Sep 22

Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning

arXiv:2604.08974v2 Announce Type: replace Abstract: Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection an...

By Lorenzo Jaime Yu Flores, Cesare Spinoso di-Piano, Jackie Chi Kit Cheung

llms nlp fine-tuning safety

One email a morning, machine-written

Related stories

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models

The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning