arXiv Machine Learning August 5, 2026 By Anton Rasmussen, Hong Qin

Quantization Effects on Biomedical LLM Reliability

Read the original on arXiv Machine Learning →

arXiv:2608. 03854v1 Announce Type: new Abstract: When decoder language models are used as classifiers, predicted class probabilities depend on implementation choices, including the prompt template, verbalizer (label-to-token mapping), and scoring rule, that are rarely treated as experimental variables.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 24

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.

By Matthew ffrench-Constant, Daniel Yang, Xinmeng Huang, Sanyam Kapoor

llms benchmarks

arXiv AI

Jun 2

Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605. 27752v2 Announce Type: replace Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.

By Hankyeol Kim, Pilsung Kang

llms benchmarks

arXiv AI

Aug 11

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.

By Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam, Quan Z. Sheng

llms nlp

arXiv AI

Aug 10

Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605. 27752v3 Announce Type: replace Abstract: Is verbalized confidence better calibrated than token likelihood?

By Hankyeol Kim, Pilsung Kang

llms

arXiv AI

1d ago

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

arXiv:2608. 16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation.

By Yifan Zhang, Rahmatollah Beheshti

llms nlp benchmarks safety

arXiv Machine Learning

Jun 5

Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries

arXiv:2606. 05970v1 Announce Type: cross Abstract: Large language models are increasingly used for structured extraction from clinical free-text notes, but the sensitivity of their output to upstream configuration choices is less understood than their accuracy on fixed benchmarks.

By Martin Murin

llms benchmarks

Related stories

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries