The paper introduces FinDendro, a benchmark of 2,226 dated U.S. facts from 1999 to 2026, designed to test large language models’ (LLMs) temporal self‑knowledge in financial backtesting. Each fact is paired with four prompts that probe the model’s recall, prediction, and self‑reported knowledge, with controls ensuring chance accuracy near 0.27. Results reveal that most frontier models understate their knowledge, with GPT‑5 correctly answering 76% of facts yet claiming to know only 38%, and that declared dates largely eliminate prior‑knowledge claims without affecting accuracy.
arXiv:2605.24564v2 Announce Type: replace Abstract: Backtesting large language models (LLMs) on historical financial data is unreliable when their pre-training data include the evaluated events. An L...
arXiv:2608. 16795v1 Announce Type: cross Abstract: Systems that generate scientific research questions are evaluated today by expert scores, LLM-as-judge ratings, or curated case studies -- all subjective, none falsifiable.
FinForesight is a diagnostic backtesting benchmark designed to evaluate deep research systems for financial forecasting. It tests four forecasting tasks—valuation, return ranking, event direction, and future financial fundamentals—across Chinese, U.S., and Hong Kong markets, while also including two diagnostic tasks for counterfactual reasoning and financial calculation. The framework limits evidence to what was available at the forecast date and compares pretraining-era, post-cutoff, and live evaluations to assess how well controlled backtests predict future performance.
Large language models (LLMs) can synthesize financial narratives but may express high confidence when evidence is sparse, stale, or contradictory. This failure is especially consequential in forecasting, where filings, news, prices, volume, and technical signals can disagree.
arXiv:2609.36914v1 Announce Type: new Abstract: Post-training has been shown to significantly improve language models' performance on tasks with verifiable outcomes, including mathematical reasoning,...
arXiv:2606. 24950v1 Announce Type: new Abstract: Financial decision-making is contextual: forecasting prices, valuing companies, and assessing event exposure weigh price history, accounting fundamentals, macroeconomic regime, and contemporaneous text.
arXiv:2607. 12248v1 Announce Type: cross Abstract: Large pretrained time-series models such as TimesFM are attractive for financial forecasting, but raw directional accuracy is a misleading scoreboard in equity markets.
arXiv:2607. 23514v1 Announce Type: cross Abstract: Multimodal automated fact-checking (MAFC) verifies claims by retrieving and reasoning over external evidence.
arXiv:2607.11414v2 Announce Type: replace Abstract: Large language models (LLMs) in financial applications fail most consequentially when they are confidently wrong. Hedged, uncertain answers invite...
arXiv:2607. 18867v1 Announce Type: new Abstract: Large language models leak parametric knowledge of realized outcomes into historical financial decision tasks.
arXiv:2605. 03460v3 Announce Type: replace Abstract: Time series (TS) reasoning models (TSRMs) have shown promising capabilities in general domains, yet they consistently fail in the financial domain, which exhibits unique characteristics.
Hedge-Bench is a benchmark comprising 92 real-world environments derived from hedge fund analysts' reasoning traces, designed to test AI agents’ ability to forecast future events from incomplete information. Agents are evaluated on joint success—correct forecasts with over 60% research coverage—across 11 frontier models, achieving rates between 2.54% and 31.52%. Training Muse‑Glimmer with GRPO on 165 environments improves coverage on held‑out tasks and demonstrates generalization to out‑of‑distribution evaluations.