FM‑Bench is a new benchmark that tests large language model agents on long‑horizon decision making by having them run a football club for 20 in‑game years. The agent must manage a squad, trade players, negotiate contracts, invest in facilities and youth, set lineups, and respond to a board that can fire it, all using 26 tools and roughly 340–400 decision stops, with a deterministic engine producing a final score without human or LLM judges. The benchmark includes a solo track where each of 15 frontier models competes against a frozen scripted world, and an Arena track where the same models plus a scripted anchor share one 20‑year world, allowing the first head‑to‑head evaluation at this scale.
whyItMatters":"FM‑Bench provides a rigorous, large‑scale test of sustained, cumulative decision‑making in language‑model agents, revealing that managerial strategy—not computational scale or vendor—drives performance over long horizons."
By Tianyou Wang, Chongyang Gao, Kezhen Chen, Chen Dong, Yinghao He, Donghan Li, Wangcheng Xu, Hongjiu Zhang, Chi Li
arXiv:2607. 17765v1 Announce Type: cross Abstract: We introduce WC2026-Agents, a benchmark and dataset for evaluating large language models (LLMs) as autonomous forecasting agents on real, future events.
By Jiacheng Ding, Cong Guo, Jason Xu
SpireBench evaluates long‑horizon strategic play by having large language models play the full commercial deck‑builder Slay the Spire, a game where early choices determine long‑term success. Across 20 LLMs at the highest difficulty, frontier models beat heuristic baselines and match average human performance but lag far behind experts, especially as runs lengthen. The benchmark reveals that strong models excel in early fights but misjudge risk over a run, and that their performance depends partly on prior game knowledge rather than on memories written during play.
By Scott C. Lowe, Sander W. Keemink
arXiv:2607. 02255v1 Announce Type: new Abstract: Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see.
By Xiangchen Cheng, Yunwei Jiang, Jianwen Sun, Zizhen Li, Chuanhao Li, Xiangcheng Cao, Yihao Liu, Fanrui Zhang, Li Jin, Kaipeng Zhang
StoreBench is a live‑commerce environment where an autonomous agent runs a mid‑size online apparel store on a production‑grade backend, facing continuous customer orders, supplier price changes, and market shocks. The agent uses the same 29 merchant tools as a human operator, with a windowed operation budget that ties simulated time to actions, ensuring that model latency does not affect the simulation. Seven large language models were evaluated across 11 scenarios of 30–45 days and a full simulated year, but none matched the scripted smart‑triage policy, and human experts outperformed all models.
By Daksh Raghuvanshi, Ved Vedere, Yifan Wang
arXiv:2606. 29771v1 Announce Type: new Abstract: LLM agents are increasingly cast as autonomous portfolio managers, and benchmarks have moved from financial question-answering to sequential trading.
By Bo Qu, Mingguang Chen
Football score forecasting combines a strong statistical core with a difficult contextual edge. Dynamic Poisson-family models estimate team strength, expected goals, and coherent score probabilities, but do not directly understand roles, tactical matchups, motivation, or how a first goal changes behaviour.
The paper presents a continuous Dyna loop that trains a multi‑agent world model for a ten‑hero MOBA game and evaluates the resulting policy in the real game. By using imagined episodes and real‑game data to continuously refine the model, the policy achieves a 70.2% win rate as radiant on unseen seeds, a dramatic improvement over dream‑only training. The study highlights that internal dream metrics can miss model exploitation, that the world model must be repaired on the policy’s own games, and that the policy’s success stems from macro‑level pressure rather than precise micro‑tactics.
By Jordy Kieto
The paper introduces Conservative Meta-Agent Search (CMAS), a method for adapting agent architectures in non‑stationary environments. CMAS retrieves archived designs, proposes typed graph edits, and evaluates them through paired shadow replay with variance‑penalized scoring and a simulated canary stage before promotion. Using the MetaInvest‑Bench simulator, CMAS reduces surrogate dynamic‑oracle regret and improves held‑out utility across 96 financial streams, while maintaining low false‑promotion rates and modest inference cost.
By Ali Atiah Alzahrani
arXiv:2610.18354v1 Announce Type: new
Abstract: Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capabil...
By Arastun Mammadli, Herbert Woisetschl\"ager, Jawad Fayaz, Shiqiang Wang
arXiv:2608. 05030v1 Announce Type: new Abstract: Football score forecasting combines a strong statistical core with a difficult contextual edge.
By Shaopeng Liang
arXiv:2609.32805v2 Announce Type: replace
Abstract: Long tasks produce more history than an LLM agent can hold in its context, and more than it uses reliably even when the history fits. A growing lin...
By Bingyu Shen, Boyang Li