arXiv Computer Vision

Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents

arXiv Computer Vision
Oct 27

DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis

arXiv:2603.01433v3 Announce Type: replace Abstract: We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanni...

By Zengqi Zhao, Weidi Xia, En Wei, Yan Zhang, Jane Mo, Tiannan Zhang, Yuanqin Dai, Zexi Chen, Yiran Tao, Simiao Ren
arXiv Computation and Language
Oct 12

Measuring Artifact Reliance in Machine-Generated Text Detectors

The study evaluates machine‑generated text detectors on scientific literature, finding that many detectors perform near perfectly on certain benchmarks due to superficial formatting cues rather than genuine linguistic differences. A logistic regression using 27 superficial features achieved macro‑F1 scores between 0.833 and 0.997, outperforming other detectors on arXiv subsets. The most powerful cue was the PDF extraction artifact—human texts preserve line breaks while generated texts appear as a single block—allowing classification at 0.996 AUC. By introducing format counterfactuals and normalizing formatting, the authors demonstrate that re‑wrapping machine‑written abstracts can flip 20–27% of detector decisions, reducing the generalization gap and improving out‑of‑domain performance.

By Anastasia Voznyuk, Andrey Grabovoy
arXiv Computation and Language
Oct 16

Multimodal AI Detection for Math-Heavy Scientific Text

The paper introduces SnapJudge, an image‑based AI detection system for math‑heavy scientific text, trained on screenshots of LaTeX‑generated PDFs. It shows that existing text‑only detectors like Pangram 4 lose accuracy as mathematical content increases, while fine‑tuned LaTeX detectors recover performance. SnapJudge achieves 92–93% detection rates at a 0.1% false‑positive rate, outperforming both Pangram 4 and text‑based models.

By Yixuan Huang, Danish Pruthi, Bhuwan Dhingra
arXiv AI
Oct 27

GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics

GPT4o-Receipt is a benchmark dataset of 1,235 receipt images that pairs GPT‑4o‑generated receipts with authentic ones from established datasets. The study evaluates five state‑of‑the‑art multimodal LLMs and a 30‑annotator crowdsourced perceptual study, revealing that humans are better at spotting AI artifacts yet perform worse at binary detection than models like Claude Sonnet 4 and Gemini 2.5 Flash. The paradox is explained by the fact that the dominant forensic signals—arithmetic errors—are invisible to visual inspection but can be quickly verified by LLMs.

By Yan Zhang, Simiao Ren, Ankit Raj, En Wei, Dennis Ng, Alex Shen, Jiayu Xue, Yuxin Zhang, Evelyn Marotta
arXiv Computation and Language
Oct 28

DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding

DocXEval is the first comprehensive framework that adapts existing explainable AI methods and evaluation metrics to multimodal document understanding, covering text, layout, and image modalities. The authors systematically evaluate 13 XAI methods across 3 tasks, 8 datasets, 4 models, and 17 metrics, finding that Occlusion consistently performs best in faithfulness and interpretability. The study also provides meta‑evaluation insights, diagnostic analyses of modality contributions, spatial attribution patterns, and potential dataset biases, offering a benchmark and practical guide for future research.

By Saifullah Saifullah, David Dembinsky, Stefan Agne, Andreas Dengel, Sheraz Ahmed