arXiv AI
Oct 9

AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks

AgentHorizon is a benchmark comprising 1,373 computer‑use tasks with instruction‑trajectory pairs collected from 166 hours of human‑recorded activity across three operating systems. The benchmark uses a paired design that swaps instructions to create negative tasks, testing judges’ ability to distinguish successful trajectories from those that completed a similar but incompatible request. Eleven judges were evaluated, with GPT‑5.5 achieving the highest balanced accuracy of 80.9% on the full trajectory subset, while tool‑use benefits some models but harms others, revealing large variability in judges’ performance.

By Xing Han L\`u, Dheeraj Vattikonda, Sina Hajimiri, Fatemeh Pesaran Zadeh, Parishad BehnamGhader, Ghazwa Darwiche, Amirhossein Kazemnejad, Christopher Pal, Alexandre Drouin, Siva Reddy
arXiv AI
Jul 20

Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving

arXiv:2607. 15621v1 Announce Type: cross Abstract: Large language models bring instruction following and scene reasoning to end-to-end driving, but their inference latency collides with the control rate a vehicle requires.

By Yun Li, Jiachen Gong, Simon Thompson, Ehsan Javanmardi, Qunli Zhang, Zifan Zeng, Shiming Liu, Peng Wang, Zixuan Guo, Manabu Tsukada
arXiv Computer Vision
Oct 23

Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models

The paper introduces a training‑free method called “Track, Then Bind” for video‑language models to reason about object identity over time. It separates semantic identity binding from temporal identity maintenance by using a promptable video tracker to follow candidate objects and a frozen vision‑language model to bind an object to a track based on visible evidence. The approach achieves state‑of‑the‑art performance on several benchmarks, outperforming fine‑tuned trajectory models and other training‑free agents on tasks such as VET‑Bench, Perception Test cup games, and TOC‑Bench.

By Di Yang, Xiaochang Li, Muyang Liu, Yufei Wang, Yanhai Xiong