arXiv:2606. 29705v1 Announce Type: new Abstract: Data, as the fundamental substrate of modern intelligence, has greatly driven the development of current foundation models.
arXiv:2511. 07332v2 Announce Type: replace-cross Abstract: Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements.
arXiv:2610.02320v1 Announce Type: cross Abstract: Computer-use agents need to reliably ground action targets in complex desktop scenes, where multiple applications, overlapping windows, and visually...
The paper introduces MHVF, a training‑free framework for GUI grounding that uses multi‑hypothesis exploration and local visual verification with a single pretrained vision‑language model. MHVF generates spatial hypotheses from full‑screen predictions, refines them through zoomed re‑localization, and verifies candidate agreement with the instruction to resolve ambiguity. Experiments on eight benchmarks with five models show significant accuracy gains, notably improving UI‑Venus‑Ground‑7B by 4.3 pp on Mind2Web and raising Qwen2.5‑VL‑7B’s ScreenSpot‑Pro accuracy from 23.6 % to 53.6 %.
arXiv:2511. 00810v4 Announce Type: replace-cross Abstract: Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regions on the screen.
arXiv:2607. 10891v1 Announce Type: new Abstract: Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interfaces (GUIs).
TikZilla is a new approach to generating TikZ code from textual descriptions, built on a larger, higher‑quality dataset called DaTikZ‑V4 that includes LLM‑generated figure descriptions. The method uses a two‑stage pipeline: supervised fine‑tuning of small Qwen models (3B and 8B) followed by reinforcement learning with an image encoder that provides semantically faithful reward signals. Human evaluations show that TikZilla outperforms its base models by 1.5–2 points on a 5‑point scale, beats GPT‑4o by 0.5 points, and matches GPT‑5 in image‑based tests while remaining much smaller.
arXiv:2606. 24515v1 Announce Type: new Abstract: Computer-Use Agents (CUAs) execute high-level user goals by perceiving and acting directly within graphical user interfaces.
arXiv:2608. 03979v1 Announce Type: cross Abstract: We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration.
arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...
arXiv:2609.16059v1 Announce Type: cross Abstract: Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fin...
The paper introduces a label‑free test‑time training approach for GUI grounding, leveraging confidence patterns in coordinate tokens rather than full‑sequence confidence. It proposes Confidence‑Anchored Learning (CAL) to filter pseudo‑labels and assign distance‑based binary rewards, and extends this to Confidence‑Anchored Negative Learning (CANL) which optimizes solely on negative samples to avoid noisy positives. Experiments show that CANL‑7B achieves 92.1% on ScreenSpot‑V2 and 33.8% on ScreenSpot‑Pro, improving the base model by 8.9%.