arXiv AI
RetryBench: Separating Retry Permission from Recovery Outcomes in GPT Tool Use
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 1
NAQD Env: A benchmark for selective withdrawal in language agents
The paper introduces NAQD‑Env, a synthetic benchmark designed to test language agents’ ability to selectively withdraw and resume actions when new evidence, permissions, or stop instructions arise. It evaluates models against a deterministic reference policy across eleven dependency families, measuring policy agreement, task value, withdrawal, resumption, and event reporting. Experiments on 350 scenarios show low withdrawal recall, no valid resumption, and limited policy alignment, highlighting the need to treat selective withdrawal as a distinct reliability component.
arXiv Computation and Language
Sep 23
FIRE: Failure-Informed Runtime Engineering for Reliable Language-Model Agents
arXiv:2609.26048v1 Announce Type: cross Abstract: Language-model agents often reach a working solution and then fail to consistently deliver it. We study runtime policies: targeted natural-language i...
arXiv AI
Jul 28
SAGE: Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AI
arXiv:2607. 22926v1 Announce Type: new Abstract: High-impact generative AI makes catastrophic misuse a lifecycle-control problem, not merely a prompt-filtering problem.
arXiv AI
Aug 24
AID-Guard: Stateful Authorization for Delegated Agent Effects
arXiv:2608.21159v1 Announce Type: cross Abstract: Tool-using AI agents turn delegated tasks into provider effects, yet authorization often ends at admission while provider state, delivery, retry, and...
arXiv AI
Oct 26
Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows
The paper investigates how AI agents can reliably recover from changes in authorization while executing tasks that involve external tools. It defines distinct obligations—checkpointing, permission to dispatch, evidence of external effects, and permission to reconcile—and evaluates them through a finite conformance study on 336 histories across multiple hosts, schedules, and provider contracts. The study demonstrates that all paired host cells produce identical outcomes and that various retry strategies (cached, revalidation, stable-key) successfully pass the majority of histories, while a component study of InfinityOS further isolates the impact of authority, contracts, and observations on recovery.
arXiv Machine Learning
Jul 1
Falsification, Not Exposure: An Internally Preregistered Placebo-Controlled Decomposition of Self-Repair Feedback in Frozen Small Code Models
arXiv:2606. 31511v1 Announce Type: cross Abstract: In deployment settings where retraining is infeasible, small frozen code models are routinely asked to repair a failed program after seeing their own failing output, usually treated as a retry mechanism.