← Back to all news
arXiv AI By Yi Wang, Haopeng Zhang, Chengxiang Huang, Rui Dai, Kaikui Liu, Piotr Koniusz, Xiangxiang Chu

LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering

Read the original on arXiv AI →

LoopArena is a benchmark that evaluates how effectively a model (the Controller) can guide a separate coding agent (the Worker) through long-running tasks by monitoring progress, assigning work, and deciding when to stop. It consists of three settings: Type I with 160 low‑cost control questions, Type II with 80 task slices, and Type III with 80 full tasks. In full tasks, the best observed strict success rate is 36.25%, and Controllers can reduce inference cost by an average of 60.6% while maintaining a strong correlation in performance ranking.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 28

Same Model, Different Harness: Different Coding-Agent Results

The paper investigates how altering the harness—specifically the way a coding agent manages context and tool outputs—affects performance when the underlying model and task remain unchanged. Two harness configurations were compared on three coding benchmarks: a control that preserves the full conversation in order, and a treatment that mechanically shortens older tool results to keep the context tight. Across all benchmarks, the treatment increased the mean per‑task fail‑to‑pass fraction and, in some cases, the number of complete solutions, demonstrating that the harness itself can significantly influence a frozen model’s effectiveness.

By Sydney Lewis
agentsbenchmarks
More like this →