FinForesight is a diagnostic backtesting benchmark designed to evaluate deep research systems for financial forecasting. It tests four forecasting tasks—valuation, return ranking, event direction, and future financial fundamentals—across Chinese, U.S., and Hong Kong markets, while also including two diagnostic tasks for counterfactual reasoning and financial calculation. The framework limits evidence to what was available at the forecast date and compares pretraining-era, post-cutoff, and live evaluations to assess how well controlled backtests predict future performance.
arXiv:2608. 03259v1 Announce Type: cross Abstract: As time-series foundation models have emerged, the need for benchmarks that can evaluate their forecasting ability in meaningful ways has become increasingly important.
arXiv:2609.36914v1 Announce Type: new Abstract: Post-training has been shown to significantly improve language models' performance on tasks with verifiable outcomes, including mathematical reasoning,...
Large language models (LLMs) can synthesize financial narratives but may express high confidence when evidence is sparse, stale, or contradictory. This failure is especially consequential in forecasting, where filings, news, prices, volume, and technical signals can disagree.
arXiv:2608.03259v3 Announce Type: replace-cross Abstract: Time-series foundation models (TSFMs) are increasingly evaluated on broad benchmarks that apply uniform conventional forecast evaluation metr...
Hedge-Bench is a benchmark comprising 92 real-world environments derived from hedge fund analysts' reasoning traces, designed to test AI agents’ ability to forecast future events from incomplete information. Agents are evaluated on joint success—correct forecasts with over 60% research coverage—across 11 frontier models, achieving rates between 2.54% and 31.52%. Training Muse‑Glimmer with GRPO on 165 environments improves coverage on held‑out tasks and demonstrates generalization to out‑of‑distribution evaluations.
arXiv:2607. 16229v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as components of agentic systems that observe, plan, and act.
WhatIf-Bench is a semi‑synthetic benchmark that tests how well time‑series forecasters use covariates to predict a target’s response to future events. It generates 37,665 tasks from mechanistic simulators of data centers, power grids, road traffic, and water networks, varying covariate numbers, historical events, and event length. Evaluation shows large‑language models outperform the best time‑series foundation models, with all models performing well on replayed events but struggling on novel covariate settings, highlighting generalization as the main challenge.
The paper presents a controlled benchmark of nine deep‑learning architectures for multi‑horizon financial forecasting, using 918 runs across 12 assets and two forecast horizons. It employs a five‑stage evaluation pipeline that isolates architectural effects from hyperparameter tuning, random seeds, and asset characteristics. Results show ModernTCN and PatchTST consistently outperform others, with ModernTCN achieving the lowest RMSE in 75% of asset‑horizon points and a statistically significant global ranking advantage.
arXiv:2605.03762v2 Announce Type: replace Abstract: Forecasting is a composite capability encompassing information gathering, evidence integration, situational assessment, and decision-making. Large...
arXiv:2608. 14106v1 Announce Type: cross Abstract: When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation.
arXiv:2605.24564v2 Announce Type: replace Abstract: Backtesting large language models (LLMs) on historical financial data is unreliable when their pre-training data include the evaluated events. An L...
arXiv:2606. 24950v1 Announce Type: new Abstract: Financial decision-making is contextual: forecasting prices, valuing companies, and assessing event exposure weigh price history, accounting fundamentals, macroeconomic regime, and contemporaneous text.