arXiv AI August 21, 2026 By Jai Kumar Sharma, Amartya Dutta

Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?

arXiv:2608. 19376v1 Announce Type: cross Abstract: Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Aug 19

Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?

Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs). We audit this practice under de...

llms multimodal safety

arXiv Machine Learning

Jul 21

The Label Complexity of Class-Conditional Coverage under Distribution Shift

arXiv:2607. 18088v1 Announce Type: new Abstract: Standard evaluation of many recognition systems contains distribution shift by construction, since benchmarks place disjoint conditions in the training and test splits.

By Weijia Han, Lisha Qu

benchmarks

arXiv Machine Learning

Aug 3

Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift

arXiv:2607. 28696v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can retain high observed marginal coverage after clinical shift while substantially under-covering an individual disease class.

By Mushir Akhtar, M. Tanveer

llms multimodal

arXiv AI

Jun 10

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.

By Bruce Changlong Xu, Lan Wu, Alexander Ryu

llms multimodal benchmarks

arXiv Machine Learning

Sep 2

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...

By Genpei Zhang

llms multimodal benchmarks

arXiv Machine Learning

Aug 26

Focal Calibration Loss: Controlling Posterior Distortion in Deep Neural Classifiers

arXiv:2410.18321v3 Announce Type: replace Abstract: Confidence calibration matters wherever a classifier's probabilities, not just its labels, are consumed downstream. We study Focal Calibration Loss...

By Wenhao Liang, Liangwei Zheng, Wei Zhang, Weitong Chen

llms safety

One email a morning, machine-written

Related stories

Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?

The Label Complexity of Class-Conditional Coverage under Distribution Shift

Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

Focal Calibration Loss: Controlling Posterior Distortion in Deep Neural Classifiers