arXiv AI

Policy as Code: A Coroutine-Bridge Harness for Fast-Reasoning Reliability on CAR-bench

The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.

arXiv Computation and Language
Nov 5

LLM Agents Already Know When to Call Tools -- Even Without Reasoning

The paper introduces When2Tool, a benchmark of 18 environments that categorizes tasks by whether a tool call is necessary, covering computational scale, knowledge boundaries, and execution reliability. It evaluates two training‑free baselines—Prompt‑only and Reason‑then‑Act—finding they inadequately control tool usage. By probing hidden states, the authors discover that models can predict tool necessity with high accuracy, leading to the Probe&Prefill method that reduces unnecessary tool calls by 38% with minimal accuracy loss.

By Chung-En Sun, Linbo Liu, Ge Yan, Zimo Wang, Tsui-Wei Weng
arXiv AI
Nov 6

The Last Mile to the Real World: Agent-Native Primitives as Reusable Building Blocks for Solving Real-World Tasks

The paper introduces agent‑native primitives—reusable building blocks that embed a language model within a resource to apply it contextually—and HEELO, a harness‑engineering framework that composes these primitives into task‑specific graphs, verifies outcomes, and iteratively refines or adds primitives. Using eight primitive families, the authors demonstrate significant performance gains across multiple benchmarks, raising Terminal‑Bench 4.0 resolution from 37.3% to 55.5% and improving pass rates on ToolBench and GAIA‑Text‑103. When integrated with HEELO, Claude Opus 5.5 and GPT‑6.1 Sol achieve higher completion counts, faster runtimes, and lower costs on 46 real‑world tasks.

By Haibo Jin, Peng Kuang, Xiaopeng Yuan, Lucheng Fu, Jerry Wang, Xucheng Yu, Yaoning Yu, Dehao Wu, Runguo Li, Colin Su, Jingxuan Sun, Xuyang Liu, Haojing Luo, Haohan Wang
Hugging Face Trending Papers
Nov 5

The Last Mile to the Real World: Agent-Native Primitives as Reusable Building Blocks for Solving Real-World Tasks

The paper introduces agent‑native primitives—reusable building blocks that embed a resident LLM into resources, enabling them to apply themselves in context and communicate with other primitives. It presents HEELO, a harness‑engineering framework that composes these primitives into task‑specific graphs, verifies outcomes, refines failures, and incorporates new primitives. Experiments show significant performance gains across multiple benchmarks, with HEELO‑enhanced agents completing more real‑world tasks faster and at lower cost.

arXiv AI
Oct 26

Benchmarks Are Surprisingly Compaction-Friendly

The paper investigates how context compaction—losing information to free up a language model’s context window—affects performance on long‑horizon agentic tasks. Using the OpenCode scaffold and five compaction strategies, the authors evaluate three frontier models on PaperBench, ProgramBench, and PostTrainBench, finding that compaction thresholds largely do not hurt performance for 1M‑context models, but can degrade GPT‑5.3‑Codex on certain benchmarks. The study also shows that compaction can silently drop critical task constraints, especially on ProgramBench and real user‑agent sessions from SWE‑chat, yet current benchmarks remain largely compaction‑friendly because objectives are fixed and progress is stored externally.

By Jeanne Salle, Hardik Bhatnagar, Matthias Bethge, Sahar Abdelnabi, Maksym Andriushchenko
arXiv AI
Oct 1

Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents

Mid‑Harness proposes a test‑time compute strategy that samples and verifies candidate actions before execution, keeping the underlying generator and harness unchanged. Experiments show that with a strong verifier, sampling more actions significantly boosts success rates—e.g., a GPT‑5.6 verifier raises Pass@1 from 50.00 % to 68.03 % on TerminalBench‑Lite using eight samples. The approach also improves performance across various models, benchmarks, and harnesses, demonstrating that action scaling is a promising target for enhancing terminal agent reliability.

By Minki Kang, Ryo Hachiuma, Shaokun Zhang, Subhashree Radhakrishnan, Yonggan Fu, Jindong Jiang, Mingjie Liu, Ehsan Hosseini-Asl, Yi Dong, Yu-Chiang Frank Wang, Byung-Kwan Lee
arXiv AI
Sep 25

Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races

The paper investigates how tool‑using language‑model agents can safely commit changes to infrastructure when external state may change between read and commit. By distinguishing invalidating races from predicate‑preserving and irrelevant ones, the authors evaluate three commit‑time guard granularities—global epoch, read‑set version, and semantic commit predicate—using a deterministic simulator and three quantized model families. The study finds that only the complete predicate guard consistently eliminates unsafe commits, while freshness‑based guards block a large proportion of benign races and model‑side signals fail to replace precise semantic enforcement.

By Zihao Zheng, Jiayu Long, Baichuan Li, Junyi Yao
arXiv AI
3d ago

EvoX: Rethinking the Agent Harness as a Runtime Policy

EvoX is a Rust-based, multi-model agent runtime that treats the agent harness as an operational policy, complementing the foundation model’s semantic policy. It introduces three mechanisms—Action Realization, Execution Continuity, and Outcome Grounding—to align model decisions with successful execution. In evaluations across five benchmarks, six foundation models, and six harnesses, EvoX achieves the highest success rate of 56.2%, outperforming other harnesses by about 3 percentage points and showing notable gains on GLM‑5.3 and Kimi K3.

By Haoyang Zhang, Qumeng Sun, Xiao Zhang, Xiang Liu, Zhidong Fan, Junjie Wang