arXiv:2606. 30697v1 Announce Type: cross Abstract: Current operating systems expose interfaces optimized for human users but not for AI agents.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 14443v1 Announce Type: new Abstract: Computer-use agents are becoming capable software operators, but their interface to desktop applications is still often a brittle motor layer: they look at screenshots, predict coordinates, click, and hope that the visible state changed as intended.
The paper introduces ASIL, an Agent‑Software Interaction Layer that replaces traditional screenshot‑and‑click interfaces with structured JSON observations and code‑executable semantic actions. ASIL is implemented across 15 applications and evaluated on 300 single‑application and 80 multi‑application tasks, achieving over 80% success with fewer than five actions per task. The structured interface also improves training efficiency, boosting performance of Qwen models from 58–66% to 72–80% with small‑scale supervised fine‑tuning and further gains with on‑policy reinforcement learning.
arXiv:2608. 09944v1 Announce Type: cross Abstract: Modern web interfaces are increasingly difficult to use with screen readers, particularly when pages update dynamically or hide important structure behind visual layout.
The study evaluates computer-use agents (CUAs) for blind users by conducting a three‑week diary study with eight participants using the OLLA prototype. Across 1,258 commands in 12 desktop applications, GPT‑5 achieved the highest success rate of 52.5%, while analysis uncovered failures in grounding, planning, constraint‑tracking, and termination. Interviews highlighted additional needs beyond automation for blind users.
The paper introduces a safety‑bounded gateway that translates IEEE 11073 Service‑Oriented Device Connectivity (SDC) into the Model Context Protocol (MCP) for medical AI agents. It exposes device metrics, alarms, context references, and semantic metadata as read‑only resources, while representing selected action affordances as policy‑validated dry‑run tools, ensuring that agent requests never trigger actual device operations. A Python prototype demonstrates fault and lifecycle experiments, deterministic baselines, and multi‑model agent evaluation, showing improved semantic conformity and preservation of the no‑execution boundary.
arXiv:2605. 10555v2 Announce Type: replace Abstract: As AI agents transition from research prototypes to enterprise production systems, the tool interfaces they consume remain rooted in human-oriented CRUD paradigms.