← Back to all news
arXiv AI By Xu Liu, Hanshu Yan, Taha Aksu, Quang Pham, Zhongzheng Qiao, Jun Hao Liew, Silvio Savarese, Doyen Sahoo, Junnan Li

GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation

Read the original on arXiv AI →

GIFT‑Eval‑CTX is a new benchmark for contextual forecasting that tests whether models can use verified future information to produce accurate time‑series predictions. It builds on GIFT‑Eval and contains 10,000 instances across 20 datasets, each annotated with macro‑level and micro‑level effects derived from actual future events. The benchmark evaluates traditional TSFMs, advanced LLMs, and agentic systems, showing that structured effects can substantially lower MASE for models like GPT‑5.6 Sol and Claude Opus 4.8, with MoiraiAgent achieving the best overall performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 25

LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...

By Xiaogang Xu, Jiaqi Tang, Jianmin Chen, Yingying Yan, Zhenchao Tang, Xiangxin Zhou, Xiaobin Hu, Wei Wei, Jinfeng Wu, Qifeng Chen, Lu Zhou, Jiafei Wu, Zhe Liu, Jianwei Yin, Weimin Zheng
llmsragagentsbenchmarks
More like this →
arXiv AI
Oct 19

CastFlow: Bridging Evidence Acquisition and Accurate Prediction in Agentic Time Series Forecasting

CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.

By Bokai Pan, Mingyue Cheng, Xiaoyu Tao, Zhiding Liu, Shuo Yu, Yuchong Wu, Qi Liu, Defu Lian, Enhong Chen
llmsagentsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv AI
Aug 5

CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting

arXiv:2608. 03031v1 Announce Type: new Abstract: Time series forecasting is fundamental to decision-making in complex systems, where future dynamics are influenced not only by historical observations but also by evolving contextual features.

By Xiaoyu Tao, Mingyue Cheng, Bokai Pan, Chuang Jiang, Huanjian Zhang, Tian Gao, Yaguo Liu, Qi Liu, Enhong Chen
llmsagentsreinforcement-learningfine-tuning
More like this →
arXiv Machine Learning
Oct 22

Rethinking Agentic Model Orchestration for Time Series Forecasting

The paper introduces FairAMO, a controlled protocol to evaluate agentic model orchestration (AMO) for time‑series forecasting by freezing candidate forecasts. Experiments on GIFT‑Eval show that while AMO can improve accuracy, its gains are limited by the quality and complementarity of the underlying TSFMs, with reductions in MASE below 6% relative to the best single model. Adding more candidates yields diminishing returns, and redundant members can negate benefits, indicating that historical evidence saturates quickly and that orchestration’s contribution is modest compared to the candidate models themselves.

By Tong Guan, Tianxiang Zhan, Shirui Pan, Ming Jin
agents
More like this →