The study investigates which inputs influence a language model’s financial forecasts by creating synthetic earnings announcements that independently reverse 20‑day momentum, commentary tone, and EPS surprise. Six instruction/chat checkpoint conversions from four model families (0.49‑7.61 B parameters) were tested with 2,016 prompts each, covering 384 primary scenarios. Results show that the contrast in primary EPS ranges from –0.19 to 86.80 percentage points, with signed tone and momentum contrasts often exceeding EPS effects; numerical diagnostic accuracy varied from 51.0 % to 95.8 %. The experiment measures observable input reliance and numerical recognition but does not establish internal reasoning mechanisms or return predictability.
Large language models (LLMs) can synthesize financial narratives but may express high confidence when evidence is sparse, stale, or contradictory. This failure is especially consequential in forecasting, where filings, news, prices, volume, and technical signals can disagree.
arXiv:2606. 24950v1 Announce Type: new Abstract: Financial decision-making is contextual: forecasting prices, valuing companies, and assessing event exposure weigh price history, accounting fundamentals, macroeconomic regime, and contemporaneous text.
arXiv:2609.36914v1 Announce Type: new Abstract: Post-training has been shown to significantly improve language models' performance on tasks with verifiable outcomes, including mathematical reasoning,...
arXiv:2607.11414v2 Announce Type: replace Abstract: Large language models (LLMs) in financial applications fail most consequentially when they are confidently wrong. Hedged, uncertain answers invite...
The paper argues that verbalized confidence—once viewed as overconfident and coarse—has become the preferred soft‑scoring method for LLM‑as‑a‑Judge on top‑tier proprietary models released after 2025. Experiments on SummEval, AggreFact, and HelpSteer2 across up to 18 LLMs show that log‑probabilities are no longer the best signal, and that adding an overconfidence advisory and self‑debate further improves calibration and robustness. The authors note that these enhancements incur little accuracy loss on post‑2025 models but do affect pre‑2025 ones, highlighting a compatibility shift in how confidence should be measured.
arXiv:2607. 16229v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as components of agentic systems that observe, plan, and act.
arXiv:2609.38523v1 Announce Type: cross Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle co...
The paper introduces an Availability Contract to address timing mismatches in financial NLP benchmarks, requiring explicit declaration of input boundaries, evidence of availability, outcome start, and supported claims. Applying this framework to 552 earnings calls, the authors show that shifting from a call‑start proxy to a completion proxy changes return signs in 9.4% of calls and alters quintile rankings in 28.3%. In a volatility benchmark, moving target windows forward by one session increases mean squared error by 18.1‑21.4% while leaving point rankings unchanged, illustrating that labels and losses can be sensitive to the declared outcome clock.
arXiv:2609.24677v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to make predictions from numerical time-series histories and textual events. Yet accuracy alone cann...
Large language models are increasingly used as decision aids whose probability judgments shape downstream choices. Whether those judgments carry a systematic directional tilt has been hard to detect: calibration metrics aggregate unsigned errors, and naturalistic uncertainty offers no ground-truth probability.
arXiv:2512. 23847v2 Announce Type: replace-cross Abstract: We develop a statistical procedure to detect lookahead bias in economic forecasts generated by large language models (LLMs).
The paper introduces Delta-Consistent Scoring (DCS), an annotation‑free method that uses large language model representations to produce continuous monetary‑policy stance scores. DCS jointly learns absolute stance for each FOMC statement and relative shifts between consecutive statements, enforcing consistency through a delta‑consistency objective. The approach outperforms supervised probes and LLM‑as‑judge baselines, achieving up to 71.1% accuracy on sentence‑level hawkish‑dovish classification and producing meeting‑level scores that correlate strongly with inflation indicators and Treasury yield movements.