arXiv:2608. 19376v1 Announce Type: cross Abstract: Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs).
Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs). We audit this practice under de...
arXiv:2607. 18088v1 Announce Type: new Abstract: Standard evaluation of many recognition systems contains distribution shift by construction, since benchmarks place disjoint conditions in the training and test splits.
arXiv:2607. 28696v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can retain high observed marginal coverage after clinical shift while substantially under-covering an individual disease class.
arXiv:2605.00326v2 Announce Type: replace Abstract: Safety pipelines often block content by thresholding a model's first-token unsafe score. We show that a single prompt's score is not a reliable pro...
arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.
arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...
arXiv:2410.18321v3 Announce Type: replace Abstract: Confidence calibration matters wherever a classifier's probabilities, not just its labels, are consumed downstream. We study Focal Calibration Loss...
arXiv:2606. 28598v1 Announce Type: cross Abstract: Prediction sets should have high coverage to be useful, but some coverage notions are more practically relevant than others.
The paper introduces a unified evaluation framework for assessing the physical adversarial robustness of Vision‑Language Models (VLMs) deployed at the edge. It defines a conditional attack success rate (cASR) metric that counts only images where the model still localizes objects under a gray occluder, and categorizes failures into omission, substitution, and displacement. Benchmarking shows that compact grounding VLMs can be as vulnerable as traditional object detectors, with omission being the only failure visible to downstream systems, while displacement and substitution pose potentially more dangerous risks, especially under distribution shift.
arXiv:2609.10333v1 Announce Type: new Abstract: Uncertainty estimation for medical vision--language models (VLMs) using conformal prediction has gained increasing attention due to its distribution-fr...
PairCLIP‑SWA is a single‑backbone deepfake detector that fine‑tunes an OpenCLIP ViT‑L/14 visual tower using a class‑balanced corpus created by CLIP‑similarity‑guided subsampling and then applies stochastic weight averaging over the fine‑tuning trajectory. Trained on the IEEE SP Cup 2025 (DF‑Wild‑Cup) benchmark, the weight‑averaged model achieves 97.04 % accuracy, 98.64 % AU‑ROC and a 4.07 % equal error rate—outperforming the best published three‑backbone ensemble. The study shows that averaging stabilizes validation performance, prevents misranking on disjoint test sets, and that CLIP pre‑training, rather than ensembling, is the key driver of accuracy.
arXiv:2610.33063v1 Announce Type: cross Abstract: Deployed vision models auto-accept confident predictions, so high-confidence errors propagate silently. Maximum Softmax Probability (MSP), the standa...