Task-Efficient Agent Evaluation Using Cached Traces

The paper introduces a method for efficient agent evaluation by using cached traces from task executions. It proposes summarizing each trace with an LLM and embedding the summary to capture task-relevant signals while reducing system-identifying noise. Using these representations, the authors build low-dimensional system embeddings and predict a new system’s benchmark score, achieving lower error than baseline models on SWE-bench Verified and Terminal-Bench 2.0 with only a single executed task.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag
• agents
• benchmarks