WildFakeToolBench is a benchmark suite comprising 156 executable image‑processing tools sourced from 220 public repositories across 142 domains. The suite evaluates six state‑of‑the‑art AI‑generated image detectors on 105 images processed by these tools, revealing that some tools can evade detectors with success rates up to 100%. The study demonstrates that web‑sourced tools pose significant in‑the‑wild risks and underscores the need for robust detector evaluation frameworks.
By Mingjie Li, Yukun Jiang, Yicong Tan, Yang Zhang
arXiv:2607. 13234v1 Announce Type: cross Abstract: Deepfake detectors that achieve near-perfect scores on academic benchmarks collapse on real-world content: recent in-the-wild evaluations report AUC drops of 45-50% for state-of-the-art open-source models.
By Ken Jon Miyachi, Dylan Uys
arXiv:2607. 16283v1 Announce Type: cross Abstract: The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before.
By Md Faraz Kabir Khan, Saeed Anwar, Ghulam Mubashar Hassan
arXiv:2609.10002v1 Announce Type: new
Abstract: Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and t...
By Rafael M. Mamede, Pedro C. Neto, Ana F. Sequeira
arXiv:2607. 06254v1 Announce Type: cross Abstract: Deepfake image detection is currently served by three fundamentally different paradigms: commercial APIs, zero-shot vision-language models (LLMs), and open-source detectors.
By Sharayu N. Deshmukh, Md Rashidunnabi, Nelton Tiago Gemo, Kurundkar G. D., Mahamune M. R., Nilesh K. Deshmukh
arXiv:2512.17730v2 Announce Type: replace
Abstract: Detectors of AI-generated images tend to inherit the biases of the data they are trained on: models fitted to GAN imagery learn to treat GAN-specif...
By Yichen Jiang, Mohammed Talha Alam, Sohail Ahmed Khan, Duc-Tien Dang-Nguyen, Fakhri Karray
The paper introduces LAION-Mobile, a new dataset of roughly one million smartphone images with EXIF metadata, designed to test deepfake detectors on modern computational photography content. Twelve state‑of‑the‑art detectors were evaluated on a 9,115‑image subset, revealing that none achieved an AUC above 0.624 on AI‑generated images and five performed worse than chance. The study also shows that threshold calibration on legacy data can hide high false‑alarm rates, with some detectors flagging 17–91 % of real photos when recalibrated for modern content.
By Achim von Stryk, Janis Keuper
The paper introduces LAION-Mobile, a 1‑million‑image dataset of modern smartphone photos with EXIF metadata, to test deepfake detectors in a realistic computational‑photography setting. Twelve state‑of‑the‑art detectors were evaluated on a 9,115‑image subset, revealing that none achieved an AUC above 0.624 on AI‑generated content and that many detectors’ false‑alarm rates spike when thresholds are recalibrated for modern smartphone imagery. The study shows that current detectors fail to maintain both high detection performance and low false‑alarm rates on contemporary smartphone photos, highlighting a critical gap in existing evaluation benchmarks.
arXiv:2608. 03096v1 Announce Type: cross Abstract: Recent advances in video generation models have significantly intensified the deepfake threat, yet the current deepfake video detection benchmarks remain underdeveloped.
By Pei Li, Sihan Chen, Delong Ran, Tianshuo Cong
arXiv:2609.01027v1 Announce Type: new
Abstract: Out-of-distribution (OOD) detection predicts whether a test image belongs to none of the predefined classes. To evaluate this task, benchmarks need ima...
By Ruslan Rozumnyi, Mat\v{e}j Such\'anek, Tom\'a\v{s} Voj\'i\v{r}, Kl\'ara Janou\v{s}kov\'a, Ji\v{r}\'i Matas
Recent advances in video generation models have significantly intensified the deepfake threat, yet the current deepfake video detection benchmarks remain underdeveloped. In particular, the effectiveness of image-level detectors in the video domain has not been systematically assessed.
PairCLIP‑SWA is a single‑backbone deepfake detector that fine‑tunes an OpenCLIP ViT‑L/14 visual tower using a class‑balanced corpus created by CLIP‑similarity‑guided subsampling and then applies stochastic weight averaging over the fine‑tuning trajectory. Trained on the IEEE SP Cup 2025 (DF‑Wild‑Cup) benchmark, the weight‑averaged model achieves 97.04 % accuracy, 98.64 % AU‑ROC and a 4.07 % equal error rate—outperforming the best published three‑backbone ensemble. The study shows that averaging stabilizes validation performance, prevents misranking on disjoint test sets, and that CLIP pre‑training, rather than ensembling, is the key driver of accuracy.
By Md. Nakib, Sudipto Sarkar, A. J. A. Jubayer Talukder, Mohammad Ariful Haque