arXiv AI
AgBench: Agentic AI Benchmarks for Personal AI Devices
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 18
Not All AI Agents Are Equal: Characterizing Resource and Performance Dynamics
The paper investigates how large‑language‑model (LLM) based AI agents mix latency, local resource usage, and container bottlenecks when processing user requests that involve remote LLM calls and local tool execution. By measuring three representative tasks—retrieval‑augmented question answering, web search, and software coding—the authors show that agents exhibit diverse resource dynamics, with concurrent requests revealing task‑specific bottlenecks in CPU, disk I/O, and memory. Leveraging these insights, they propose CPU‑aware tool admission and task‑aware CPU allocation, achieving up to a 5.4× speed‑up for CPU‑sensitive tasks and a 32% reduction in average latency across multiple tasks.
arXiv AI
Aug 18
From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems
arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.
arXiv AI
Jul 16
DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments
arXiv:2607. 13465v1 Announce Type: cross Abstract: LLM-based agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes.
arXiv AI
Aug 11
SkillSmith: Enhancing Locally Deployed Agents via Automatic Skill Construction and Evolution
arXiv:2608. 08037v1 Announce Type: new Abstract: LLM-based agent frameworks now act as personal assistants for multi-step tasks.
arXiv AI
3d ago
cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
arXiv:2609.40284v1 Announce Type: cross Abstract: Computer use agents (CUAs), which use graphical user interfaces (GUIs) to complete tasks on a computer, have recently surpassed human performance on...
arXiv AI
Jul 24
InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents
arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.