arXiv Machine Learning
Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 1
Rethinking the Test-Time Prompt Tuning Objective from the Perspective of Calibration
arXiv:2608.30230v1 Announce Type: new Abstract: Test-time prompt tuning (TPT) has emerged as a powerful paradigm, refining prompts for each test sample via entropy minimization (EM) over multiple aug...
arXiv AI
Jun 2
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
arXiv:2605. 30188v2 Announce Type: replace-cross Abstract: Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated.
arXiv Machine Learning
Aug 27
Sample Margin-Aware Recalibration of Temperature Scaling
The paper introduces SMART, a lightweight recalibration technique that adjusts logits based on the margin between the top two logits, called the logit gap. It uses a soft-binned Expected Calibration Error objective to balance bias and variance, enabling stable updates even with limited calibration data. Experiments across various datasets and architectures show SMART achieves state‑of‑the‑art calibration with fewer parameters than existing methods.
arXiv Machine Learning
Jul 10
LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks
arXiv:2607. 07745v1 Announce Type: new Abstract: While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge.
arXiv Machine Learning
Aug 12
Invertible Logits Transformation for Accuracy-Preserving Post-Hoc Uncertainty Calibration
arXiv:2608. 10372v1 Announce Type: new Abstract: Post-hoc calibration aligns a classifier's predicted confidences with its empirical accuracy without retraining.
arXiv Computer Vision
Aug 28
MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models
MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.