arXiv AI
Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 25
Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
The paper introduces Growing Harness, a training method that transforms recurring control logic in large language model agents into reusable executable code, reducing reliance on the model for task-specific decisions. By using strategy-free scaffolds, failure-guided code repair, and success-first gating, the approach learns a shared harness that improves performance across multiple benchmarks and model sizes. Experiments on BrowseComp-Plus and WebArena-Verified show significant gains in success rates and substantial reductions in LLM calls and inference cost compared to traditional tool‑calling agents.
arXiv AI
2d ago
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Mingbird is a local‑first agent harness designed for small open‑weight language models (2–9 B) that run on ordinary laptops. It introduces ten mechanisms—such as a byte‑level net‑zero prefill budget, a finish gate that re‑reads the task before accepting completion, and signature‑level loop detection—to address common failure modes that arise from the harness rather than the model itself. In controlled experiments on the LRAB benchmark and the $ au^2$‑bench, Mingbird achieves higher overall scores (0.886 and 0.856 respectively) compared to other harnesses, and its ablation studies show that each mechanism contributes measurable performance gains.
arXiv AI
Jun 16
Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents
arXiv:2606. 16364v1 Announce Type: new Abstract: LLM agents mis-call tools, and the natural guess is that the model failed to see the right tool in a crowded harness.
arXiv AI
4d ago
One Readout, Many Repairs: Diffusion-Guided Hierarchical Search for Tool-Agent Repair
arXiv:2609.34879v2 Announce Type: replace Abstract: Tool agents use large language models to act through external tools, yet successfully executed calls can still leave user requests unfulfilled. Too...
arXiv AI
Aug 7
EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents
arXiv:2608. 05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic.
arXiv AI
Aug 7
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
arXiv:2608. 06301v1 Announce Type: new Abstract: As LLMs are increasingly deployed within agentic systems, their capabilities depend not only on the model weights but also on the harness: the prompts, tools, control flow, memory, and orchestration code surrounding them.