arXiv AI
APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
SINGED: Correct Outputs Do Not Certify Safe Execution in LLM Agents
arXiv:2609.35889v1 Announce Type: cross Abstract: Tool-using language-model agents select and execute third-party artifacts. Different implementations can return the requested output while producing...
arXiv AI
Aug 11
ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents
arXiv:2608. 09476v1 Announce Type: cross Abstract: Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API.
arXiv AI
Jul 21
AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
arXiv AI
Aug 28
BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click
BekchiAI introduces a benchmark and platform for evaluating large language model agents. The benchmark comprises 13 tool‑using ReAct agents across seven task categories, totaling 2,057 deterministic test tasks with verifier‑checkable gold answers. The platform offers web‑based observability, token and latency telemetry, and remote run termination for live agents.
arXiv AI
Jun 9
SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.
arXiv AI
3d ago
Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows
Harness-Bench is a diagnostic benchmark that evaluates how different harness configurations affect LLM agent performance in realistic workflows. It tests 106 sandboxed offline tasks across multiple model backends, recording artifacts, execution traces, and usage statistics to analyze completion, process quality, efficiency, and failure behavior. The study finds significant variation in outcomes across model-harness pairings, indicating that agent capability should be reported at the configuration level rather than solely by the base model.