arXiv Computer Vision
Fi-ImageNet-1k: An OOD Benchmark From the Inside of the ImageNet-1k Validation Set
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
Jun 2
A Closer Look at In-Distribution vs. Out-of-Distribution Accuracy for Open-Set Test-time Adaptation
arXiv:2606. 01973v1 Announce Type: new Abstract: Open-set test-time adaptation (TTA) updates models on new data in the presence of input shifts and unknown output classes.
arXiv Computer Vision
Oct 9
Multimodal Large Language Models as Image Classifiers
The paper investigates how evaluation protocols and ground‑truth quality affect the classification performance of Multimodal Large Language Models (MLLMs). It identifies and corrects key issues such as discarded out‑of‑list outputs, inflated distractor choices, and poor open‑world mapping, and shows that design choices like batch size, image ordering, and text encoder selection significantly influence accuracy. Using a multilabel reannotation of 625 ImageNet‑1k classes (ReGT), the study finds that corrected labels can boost MLLM performance by up to 10.8%, narrowing the gap with supervised models, and demonstrates that MLLMs can assist human annotators in about half of difficult cases.
arXiv AI
Jul 21
GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification
arXiv:2607. 16283v1 Announce Type: cross Abstract: The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before.
arXiv AI
Jul 8
VendorBench-100: A Unified Cross-Paradigm Benchmark for Deepfake Image Detection
arXiv:2607. 06254v1 Announce Type: cross Abstract: Deepfake image detection is currently served by three fundamentally different paradigms: commercial APIs, zero-shot vision-language models (LLMs), and open-source detectors.
arXiv AI
6d ago
How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study
The study evaluates 16 AI‑generated image detection methods in a zero‑shot setting, testing 23 pretrained variants on 12 datasets with 2.6 million images from 291 generators. Results show no single detector consistently outperforms others, with accuracy ranging from 37.5 % to 75.0 %. The study highlights that training data alignment and generator type significantly affect performance, especially against modern commercial models, and identifies systematic failure patterns across datasets.
arXiv AI
2d ago
PairCLIP-SWA: Similarity-Guided Balancing and Stochastic Weight Averaging for Generalized Deepfake Image Detection with OpenCLIP
PairCLIP‑SWA is a single‑backbone deepfake detector that fine‑tunes an OpenCLIP ViT‑L/14 visual tower using a class‑balanced corpus created by CLIP‑similarity‑guided subsampling and then applies stochastic weight averaging over the fine‑tuning trajectory. Trained on the IEEE SP Cup 2025 (DF‑Wild‑Cup) benchmark, the weight‑averaged model achieves 97.04 % accuracy, 98.64 % AU‑ROC and a 4.07 % equal error rate—outperforming the best published three‑backbone ensemble. The study shows that averaging stabilizes validation performance, prevents misranking on disjoint test sets, and that CLIP pre‑training, rather than ensembling, is the key driver of accuracy.