arXiv AI August 13, 2026 By Ruoxi Zhao, Maziar Raissi

Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs

arXiv:2608. 11232v1 Announce Type: cross Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs require ground truth from actual code execution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 3

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

arXiv:2605. 28556v2 Announce Type: replace Abstract: As agent capabilities advance, existing benchmarks, such as $\tau^2$-Bench, are becoming increasingly saturated.

By Tomer Keren, Nitay Calderon, Asaf Yehudai, Yotam Perlitz, Michal Shmueli-Scheuer, Roi Reichart

llms agents benchmarks

arXiv AI

Jul 24

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.

By Jehyeok Yeon, Ben Rank, Maksym Andriushchenko

llms agents efficiency benchmarks

arXiv AI

Jul 2

PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents

arXiv:2607. 00436v1 Announce Type: new Abstract: Large language model agents are increasingly connected to scientific software, yet it remains unclear when tool access makes scientific computation more reliable rather than merely more complex.

By Ke Zhang, Sahchit Chundur, Mohammad Javad Qomi, Maziar Raissi

llms agents benchmarks

arXiv Machine Learning

Aug 4

Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark

arXiv:2608. 00106v1 Announce Type: new Abstract: Agentic systems must decide not only what answer to produce, but which reasoning and execution operations should precede it.

By Natan Vidra, Alina Kapanova, Arun Kanhai, Spurthi Setty

llms agents benchmarks

arXiv AI

Jul 28

E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios

arXiv:2607. 23722v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as agents that interact with stateful environments over multiple steps: gathering hidden information, composing tool calls, and committing state changes.

By Weihuang Zheng, Tianyuan Zou, Eileen Ye, Alphet Liu, Youyong Kong, Ya-Qin Zhang, Duran Zheng, Maxm Pan

llms agents benchmarks

arXiv AI

Jul 13

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading

arXiv:2607. 08964v1 Announce Type: new Abstract: AI agents have become capable of autonomously completing short, well-specified tasks.

By Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ninghao Liu, Lichao Sun, Haotao Mi, LeoweiLiang

agents multimodal benchmarks

One email a morning, machine-written

Related stories

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents

Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark

E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading