AgentTime is a benchmark that tests whether AI agents can manage their own runtime by working for a specified duration, predicting their wall‑clock time, and estimating elapsed time afterward. It includes 222 tasks from 18 sources across coding, computer use, agentic work, and automated research, with duration‑following experiments that append a single instruction specifying how long to work. Results show significant variability in accuracy, with some agents deviating from requested runtimes by up to 2.9×, and many agents stopping prematurely even after completing a task.
By Michael Ofengenden, Maksym Andriushchenko
StoreBench is a live‑commerce environment where an autonomous agent runs a mid‑size online apparel store on a production‑grade backend, facing continuous customer orders, supplier price changes, and market shocks. The agent uses the same 29 merchant tools as a human operator, with a windowed operation budget that ties simulated time to actions, ensuring that model latency does not affect the simulation. Seven large language models were evaluated across 11 scenarios of 30–45 days and a full simulated year, but none matched the scripted smart‑triage policy, and human experts outperformed all models.
By Daksh Raghuvanshi, Ved Vedere, Yifan Wang
arXiv:2609.00874v1 Announce Type: new
Abstract: As agentic systems become compound systems, increasingly important decisions move above task execution itself: when should a higher-level controller pr...
By Ayushi Chadha
arXiv:2606. 00198v1 Announce Type: cross Abstract: While agents are increasingly spending more resources, today agent cost is mostly measured only after execution.
By Yuxiang Lin, Zihan Wang, Mengyang Liu, Yuxuan Shan, Longju Bai, Junyao Zhang, Xing Jin, Boshan Chen, Jinyan Su, Xingyao Wang, Jiaxin Pei, Manling Li
arXiv:2608. 05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic.
By Jie Wu, Ming Gong, Feixiang Cheng, Qinqin Zhao
The paper investigates how a higher‑level controller in a hierarchical latent reasoner decides whether to preserve or replace a lower‑level strategy. Experiments across different training seeds show that learned revision timing can produce varied policies, but none surpasses a fixed, forced timing policy on the same checkpoint. The study highlights that state‑dependent control can alter actions without improving task performance, and that timing decisions are order‑sensitive and largely captured by a strong fixed schedule.
By Ayushi Chadha
arXiv:2610.09679v1 Announce Type: new
Abstract: Adaptive rollout methods for group-relative reinforcement learning typically allocate a fixed per-update budget across prompts. We instead study how to...
By Yiming Zong, Yige Wang, Xing Hu, Jiashuo Jiang, Zuo-Jun Max Shen
SCLATE is a new execution substrate that allows continual‑learning benchmarks and agents to share a single event scheduler via adapters, enabling tasks, session events, and memory consolidation to run on a compressed, real‑time timeline. It also functions as a rollout engine that records every model call’s tokens and log probabilities without modifying the agent’s harness or memory. Using SCLATE, the authors ported seven benchmarks, compared ten harness‑memory configurations across ten models, and demonstrated that post‑training Qwen3.5‑4B can effectively leverage both harness and memory, improving performance on multiple metrics.
By Youngmok Jung, Sirajul Salekin, Henry Tran, Javier Movellan, Zhao Huang, Manjot Bilkhu
arXiv:2609.37125v1 Announce Type: new
Abstract: Prospective memory allows an agent to retain an intention tied to a future condition, but the stored intention does not reveal whether that condition c...
By Zhengkun Di, Bin Shi, Kai Sun, Yiming Xu, Bo Dong
Deliberating takes time. In real-time settings, that time is not free.
arXiv:2609.14138v1 Announce Type: cross
Abstract: As LLM agents become integrated into increasingly complex workflows, they must continually acquire new capabilities while retaining competence on pre...
By Siddharth Sharma, Nilesh Prasad Pandey, Onat Gungor, Tajana Rosing
arXiv:2609.13739v1 Announce Type: cross
Abstract: Language-model agents are increasingly deployed through diverse harnesses that differ in system prompts, tool schemas, control loops, and trajectory...
By Hongliang Wei (Harbin Institute of Technology, Alibaba Cloud), Xiaobing Tu (Alibaba Cloud), Yinggui Wang (Alibaba Cloud), Zhengxi Liu (Alibaba Cloud), Rongkun Xue (Alibaba Cloud), Jinkui Ren (Alibaba Cloud), Xiantao Zhang (Alibaba Cloud), Debin Zhao (Harbin Institute of Technology), Xiaopeng Fan (Harbin Institute of Technology)