arXiv AI
Jun 30

CAPTCHA Solving for Native GUI Agents: Automated Reasoning-Action Data Generation and Self-Corrective Training

arXiv:2603. 23559v2 Announce Type: replace-cross Abstract: GUI agents are rapidly shifting from multi-module pipelines to end-to-end, native vision-language models (VLMs) that perceive raw screenshots and directly interact with digital devices.

By Yuxi Chen, Haoyu Zhai, Chenkai Wang, Rui Yang, Lingming Zhang, Gang Wang, Huan Zhang
arXiv Computer Vision
Sep 24

Invisible in Space, Visible in Time: Motion Vision CAPTCHA against GUI Agents

The paper introduces Motion Vision CAPTCHA (MVCAP), a new CAPTCHA framework that relies on motion-defined foreground structures to create challenges that are only solvable through temporal analysis of a dynamic background. MVCAP is implemented in three progressive levels—coherent motion, structural motion, and biological motion—and evaluated using the MVCAP-Bench, a browser-based benchmark with 600 live CAPTCHA instances. Human participants achieve 99.6% accuracy, whereas the best GUI agent scores only 16.8%, highlighting a significant human–agent perception gap and demonstrating that dynamic background camouflage is the key difficulty.

By Zeyu Zhang, Dingyi Rong, Zijian Chen, Zicheng Zhang, Xiongkuo Min, Guangtao Zhai