arXiv AI

Empty Commitments: When Agents Promise What Their Runtime Cannot Deliver

The paper introduces the concept of empty commitments, where an agent promises an action that its current tools or runtime cannot execute. It defines empty commitments using commitment semantics, identifies three failure types, and proposes an anchoring condition for realistic tool promises. The authors also outline a measurement protocol that tests follow‑up requests across five setups, each adding a different persistence affordance, with the environment either implicit or explicitly stated.

arXiv AI
5d ago

Empty Commitments: When Agents Promise What They Cannot Deliver

The paper introduces the concept of "empty commitments," promises made by AI agents that cannot be fulfilled due to the absence of necessary tools or runtime capabilities. It defines three failure types, proposes a detection checker, and evaluates it on 400 human-labeled examples and a benchmark of 293 follow‑up requests. The study shows that many open‑weight models frequently issue such empty commitments, and that simple interventions—informing the model of its runtime, adding a capability card, or running a checker loop—can significantly reduce these failures.

By Jiaqi Tang, Bingyu Shen, Lan Wei, Qing Lu, Bethel Ololade, Danny Galvis, Miles Q. Li, Bin Hu, Boyang Li
arXiv AI
Aug 12

How to Dogfood Your AI Chat Agent: A Three-Layer Evaluation Framework with Goal-Directed NPC Simulation

arXiv:2608. 09939v1 Announce Type: cross Abstract: Production teams deploying LLM chat agents face a specific quality assurance gap: existing evaluation tools test individual responses or simulate social interactions, but none systematically verify whether real users can achieve their goals through multi-turn conversation.

By Alexandre Cristov\~ao Maiorano
arXiv AI
Aug 28

Five Primitives for Governing Autonomous AI Agents at Runtime

The paper proposes five runtime primitives—discovery, identity, governance, attestation, and supply chain—to manage autonomous AI agents in enterprise settings. It argues that traditional control models fail because agents are transient, model-driven, and self‑discoverable, making runtime governance essential. The authors detail an implementation that mediates agent actions against policy, authorizes them via a per‑tenant vocabulary, and records them in a verifiable ledger, noting the associated operational costs and partial deployment status.

By Jiten Oswal, John Cadeddu
arXiv AI
Sep 4

Fresh Memory, Stale Plans: Dependency-Scoped Validation for Distributed LLM-Agent Memory

The paper introduces PlanFence, a dependency-scoped action‑validation protocol for distributed large language model (LLM) agent teams. PlanFence requires plans to cite the exact public records they rely on, and executors validate only those records that could affect the pending action, replanning or blocking if validation is incomplete. In 30 controlled live workflows, a freshness‑only executor always acted on obsolete plans, whereas PlanFence completed all tasks without invalid actions, demonstrating controlled safety and system‑cost benefits.

By Evan Chen, Shiqiang Wang, Christopher G. Brinton