arXiv Machine Learning

From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecasting

arXiv AI
Sep 2

LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting

LEAP (Likelihood Elicitation and Aggregation for Probabilistic forecasting) is a new approach that reorganizes how evidence is used in LLM-based forecasting systems. Instead of a monolithic prediction that aggregates all evidence at once, LEAP examines each evidence item separately, elicits likelihood parameters, and combines them with an explicit prior to produce a posterior distribution. The method supports continuous, single-choice, and multi-choice forecasts and has been shown to improve prediction and calibration metrics across models on a benchmark covering forecasting, information-seeking, and browsing tasks.

By Yufei Chen, Yiran Zhao, Xiaogang Xu, Qipeng Xie, Jiafei Wu, Zhe Liu
arXiv AI
Aug 25

LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...

By Xiaogang Xu, Jiaqi Tang, Jianmin Chen, Yingying Yan, Zhenchao Tang, Xiangxin Zhou, Xiaobin Hu, Wei Wei, Jinfeng Wu, Qifeng Chen, Lu Zhou, Jiafei Wu, Zhe Liu, Jianwei Yin, Weimin Zheng
arXiv AI
Sep 25

Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents

Forecast-Dojo is a replayable environment designed to benchmark and train large language model (LLM) forecasting agents. It integrates resolved prediction‑market questions with dated news, enabling agents to research events and revisit predictions at successive historical dates. The platform includes 1,568 Polymarket events, 18.8 million dated news articles, and supports repeated evaluation, training interactions, and outcome feedback, with evidence that research tools lower Brier scores across 12 tested models, though all models still lag behind historical market forecasts.

By Liqin Ye, Haorui Wang, Fardin Ahmed, Rongzhi Zhang, Yuan He, Ziyuan Lin, Yanbin Yin, Jing Peng, Michael Galarnyk, Sudheer Chava, Chao Zhang
arXiv AI
Oct 19

CastFlow: Bridging Evidence Acquisition and Accurate Prediction in Agentic Time Series Forecasting

CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.

By Bokai Pan, Mingyue Cheng, Xiaoyu Tao, Zhiding Liu, Shuo Yu, Yuchong Wu, Qi Liu, Defu Lian, Enhong Chen
arXiv AI
Oct 15

GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation

GIFT‑Eval‑CTX is a new benchmark for contextual forecasting that tests whether models can use verified future information to produce accurate time‑series predictions. It builds on GIFT‑Eval and contains 10,000 instances across 20 datasets, each annotated with macro‑level and micro‑level effects derived from actual future events. The benchmark evaluates traditional TSFMs, advanced LLMs, and agentic systems, showing that structured effects can substantially lower MASE for models like GPT‑5.6 Sol and Claude Opus 4.8, with MoiraiAgent achieving the best overall performance.

By Xu Liu, Hanshu Yan, Taha Aksu, Quang Pham, Zhongzheng Qiao, Jun Hao Liew, Silvio Savarese, Doyen Sahoo, Junnan Li
arXiv Computation and Language
Sep 14

Information Specialization and Constrained Synthesis in Multi-Agent LLM Forecasting: A Prospective Live-Study of the 2026 FIFA World Cup

The study evaluates a multi‑agent large language model system for forecasting outcomes of the 2026 FIFA World Cup. Two specialist agents—one quantitative and one news‑focused—produce forecasts that are then reviewed by a critic and combined by a meta‑agent. Results show the news specialist performs best, matching betting market accuracy, while the meta‑agent adds little beyond the specialists’ predictions.

By Julian Varghese, Lucas Bickmann, Sarah Sandmann
arXiv Machine Learning
Sep 21

Tracing the Evidence Behind Zero-Shot Time-Series Forecasting: A Source-First Taxonomy and Audit Framework

The paper argues that zero‑shot time‑series forecasting should be treated as an evidence‑access claim rather than merely a no‑parameter‑update condition. It introduces a source‑first taxonomy that distinguishes three evidence sources—frozen LLM prior reuse, parametric time‑series pretraining, and retrieval‑augmented external memory—from the architectures that implement them. The authors further outline four audit questions—task interface, forecast object and scoring, prediction‑time context, and resource budget—to make zero‑shot leaderboards transparent and comparable.

By Delun Kong, Wanyun Ling, Chenxi Liu, Ziyue Li
arXiv AI
Aug 5

CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting

arXiv:2608. 03031v1 Announce Type: new Abstract: Time series forecasting is fundamental to decision-making in complex systems, where future dynamics are influenced not only by historical observations but also by evolving contextual features.

By Xiaoyu Tao, Mingyue Cheng, Bokai Pan, Chuang Jiang, Huanjian Zhang, Tian Gao, Yaguo Liu, Qi Liu, Enhong Chen