Agent-Facing Information Design in LLM Tool Registries: A Preregistered Test of Rhetoric, Position and Structure
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
The Flow has not summarised this story yet — read it at arXiv AI.
The study investigates how sales language in tool registry descriptions influences AI agents’ tool selection. By creating paired listings that differ only in praise, verifiable specifications, or order, the authors found that stacked praise increased a tool’s pick rate by about 43 percentage points, while the first-listed tool was chosen roughly 72 points more often. Structured fields for numeric limits helped agents choose capable tools, but accompanying sales text could negate this benefit.
AgentScript‑Eval is a benchmark of 100 real tasks for Salesforce’s AgentScript DSL, testing LLMs on authoring, debugging, and optimizing conversational agents. Five models were evaluated zero‑shot, with the best achieving only 25.0% pass@3 and all scoring 0% on authoring from scratch. The study shows that model size and cost are poor predictors of success; instead, a compile‑and‑edit loop and the wording of domain guidance significantly improve performance, with the best result (37%) achieved by rewriting a skill within a live plugin.
The paper reports a case study of a large language model (LLM) coding agent tasked with building a multi‑component data system from a detailed specification. During a single session the agent introduced five defects, which were categorized by violated constraints and detection methods. The study also evaluates the agent’s retrieval‑filtering strategy on the HotpotQA benchmark, showing that filtering to a graph‑identified entity set yields higher recall than unfiltered search, with a statistically significant gap across all tested budgets.
The paper investigates how tool output formatters can mislead agents and their verifiers by omitting critical details such as search limits, result caps, or identifier namespaces, even when the underlying computation is correct. Experiments with a production Retrieval-Augmented Generation (RAG) system and other open‑source agents (OpenClaw, Hermes) show that providing complete formatter information dramatically reduces unsupported agent claims and increases verifier rejection of incorrect answers. The authors provide an audit prompt that can be applied to other systems to uncover and fix similar warped formatters.
arXiv:2610.28454v1 Announce Type: new Abstract: Agents are starting to sell to other agents. Seller agents write the listings of tools, services and products, and buyer agents read these listings whe...
arXiv:2607. 08028v1 Announce Type: new Abstract: Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context.