arXiv AI June 30, 2026 By Yanuo Ma, Ben Kereopa-Yorke, Ben Schultz

Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

arXiv:2606. 28430v1 Announce Type: cross Abstract: Benchmarks are widely used to evaluate task completion by Large Language Models (LLMs), but this approach has accumulated construction-validity problems, and a passing score may not show whether the requested task was delivered.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 21

AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows

arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.

By Tejas Singh Anand, Yuet Ying Christina Wang, Wanting Jiang, Steve Masson, Tian Zheng, Bingjie Zhou

llms agents safety

arXiv AI

Aug 11

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

arXiv:2603. 28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs.

By Han Wang, Yifan Sun, Brian Ko, Mann Talati, Jiawen Gong, Zimeng Li, Naicheng Yu, Xucheng Yu, Wei Shen, Vedant Jolly, Huan Zhang

llms benchmarks

arXiv AI

Jul 3

GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation

arXiv:2606. 22737v2 Announce Type: replace Abstract: Before letting an agent operate over real context, can you prove it used the right evidence?

By Jeffrey Flynt

llms agents benchmarks

arXiv AI

Jun 2

Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability

arXiv:2606. 01365v1 Announce Type: new Abstract: Tool-using multi-agent large language model (LLM) systems spend computation through model tokens, tool calls, retries, and code execution before producing an answer.

By Xianyou Li, Weiran Yan, Yichao Wu, Penghao Liang, Mengwei Yuan, Jianan Liu, Jing Yang

llms agents

arXiv AI

1d ago

TDD-Agent: Test-Driven Reasoning for Code Generation

arXiv:2608. 16742v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging.

By Hongyue Yu, Kefan Li, Jiakun Li, Hongzheng Chai, Yuan Yuan, Rui He, Junyi Wei

llms agents benchmarks

arXiv AI

Jun 4

Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery

arXiv:2606. 05037v1 Announce Type: cross Abstract: When an AI agent calls an API and hits a validation error, it needs more than what went wrong -- it needs what to do next.

By Arquimedes Canedo, Grama Chethan

llms agents benchmarks safety

Related stories

AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation

Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability

TDD-Agent: Test-Driven Reasoning for Code Generation

Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery