arXiv AI
PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 2
GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
arXiv:2610.00948v1 Announce Type: cross Abstract: The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termi...
arXiv Machine Learning
Oct 2
Learning Reliable GUI Agents under Imperfect Priors
arXiv:2609.39547v1 Announce Type: new Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GU...
arXiv AI
Jul 29
Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification
arXiv:2607. 25904v1 Announce Type: new Abstract: Graphical user interface task evaluation aims to determine whether a GUI agent has successfully completed a user instruction.
arXiv AI
Jul 20
SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction
arXiv:2607. 15550v1 Announce Type: new Abstract: Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences.
arXiv AI
Sep 4
Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents
The paper introduces CONFLICTGUI, a benchmark that tests GUI agents on instruction-internal and instruction‑GUI context conflicts, revealing that many agents over‑comply with infeasible instructions. To address this, the authors propose CONFLICTGUARD, an inference‑time framework that couples a feasibility verification protocol with a conditional action modulation mechanism, enabling agents to assess instruction logic and GUI evidence before acting. Experiments on five agents show that CONFLICTGUARD significantly improves conflict‑task success while maintaining normal task performance.
arXiv AI
Jun 12
ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm
arXiv:2606. 13239v1 Announce Type: cross Abstract: Existing computer-use agents remain fundamentally limited in professional software manipulation: GUI-based agents suffer from fragile visual grounding and long-horizon error accumulation, while API-basedapproaches struggle with heterogeneous protocols and inaccessible commercial interfaces.