The paper introduces FABench, a scalable backtesting environment designed to evaluate forecasting agents. FABench automatically discovers events, generates questions, and resolves outcomes for any past forecast date, allowing continuous renewal beyond model training cutoffs. In initial tests, retrieval improves Brier scores across eight models, with top agents matching or surpassing market consensus on 54 Polymarket-linked questions, yet analysis shows persistent evidence‑weighting failures that highlight room for improvement.
arXiv:2605.16358v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly applied to real-world forecasting tasks, yet evaluating their true predictive capability remain...
The paper introduces an agentic forecasting environment built on 2,100+ resolved Polymarket questions, where a language model (Qwen3.5-35B-A3B) learns to gather evidence during rollout via web search, page reading, and financial time series, all filtered to avoid post‑cutoff leaks. Training with single‑epoch GRPO and a Brier‑score reward improves calibration by 30‑40% and reduces search attempts, while the trained policy outperforms four frontier models in evidence‑based forecasting, achieving lower soft‑Brier scores at roughly 5% of the inference cost. The authors release the environment, dataset, and per‑rollout records as a reusable harness for temporal forecasting agents.
arXiv:2606. 11816v1 Announce Type: cross Abstract: Forecasting real-world events requires language-model agents to reason under uncertainty from incomplete, time-bounded information.
Forecast-Dojo is a replayable environment designed to benchmark and train large language model (LLM) forecasting agents. It integrates resolved prediction‑market questions with dated news, enabling agents to research events and revisit predictions at successive historical dates. The platform includes 1,568 Polymarket events, 18.8 million dated news articles, and supports repeated evaluation, training interactions, and outcome feedback, with evidence that research tools lower Brier scores across 12 tested models, though all models still lag behind historical market forecasts.
Time-series foundation models provide strong zero-shot forecast, but deployment for real-world applications still requires task-specific designs for model inputs and outputs. Direct inference can leav...
TSFLab is a self‑improving, agent‑agnostic workbench designed to streamline time‑series forecasting research. It equips coding agents with shared resources and tools, enabling low‑cost experimentation, reproducibility, and continuous maintenance of research artifacts. Experiments show that TSFLab boosts task completion rates by 20‑163% and reduces token usage and development time significantly compared to baseline approaches.
CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.
arXiv:2606. 02497v1 Announce Type: new Abstract: Time series forecasting has advanced rapidly, especially with the emergence of foundation models that show strong zero-shot performance on numerical extrapolation.
Hedge-Bench is a benchmark comprising 92 real-world environments derived from hedge fund analysts' reasoning traces, designed to test AI agents’ ability to forecast future events from incomplete information. Agents are evaluated on joint success—correct forecasts with over 60% research coverage—across 11 frontier models, achieving rates between 2.54% and 31.52%. Training Muse‑Glimmer with GRPO on 165 environments improves coverage on held‑out tasks and demonstrates generalization to out‑of‑distribution evaluations.
TimeRSI is a self‑improving deployment system for time‑series foundation models that automates the creation and refinement of forecasting pipelines. A coding agent translates task descriptions into executable changes, tests each revision on historical data, and uses error feedback to guide further revisions. During live deployment, an adaptive selector chooses the best pipeline based on completed forecasts under similar conditions, achieving superior performance on 198 tasks compared to all other submissions.
arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...
arXiv:2609.05905v1 Announce Type: cross Abstract: LLM agents are increasingly used for live forecasting, where they retrieve up-to-date information and produce estimates for unresolved future events....