arXiv:2605.16358v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly applied to real-world forecasting tasks, yet evaluating their true predictive capability remain...
arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...
CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.
arXiv:2602. 12147v4 Announce Type: replace Abstract: Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.
arXiv:2606. 18733v1 Announce Type: cross Abstract: Realistic coding-agent benchmarks often replay public GitHub issues and pull requests, making them vulnerable to overlap with model pretraining, fine-tuning, synthetic-data generation, or benchmark-driven model selection.
arXiv:2607. 17409v1 Announce Type: cross Abstract: We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs.
arXiv:2606. 18686v1 Announce Type: new Abstract: Forecasting benchmarks for general-purpose AI systems usually inherit the constraints of the real world: outcomes resolve slowly, tail events are rare, and counterfactual questions are difficult to score.
arXiv:2608. 03031v1 Announce Type: new Abstract: Time series forecasting is fundamental to decision-making in complex systems, where future dynamics are influenced not only by historical observations but also by evolving contextual features.
The paper introduces Alignment Forecasting, a method for predicting whether fine‑tuning a language model on a given dataset will increase specific alignment failures such as deception or sycophancy. It presents ALIGNMENTFORECASTBENCH, a benchmark of over 5,000 forecasting questions across many models, datasets, and failure modes, and shows that a simple forecasting scaffold using an LLM’s assessment of dataset bias can outperform baseline forecasters. The authors demonstrate that filtering out high‑risk training examples identified by the forecaster can improve alignment in multiple‑choice evaluations, though benefits in open‑ended conversations remain uncertain.
Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both.
arXiv:2610.07834v1 Announce Type: new Abstract: Retrieval-augmented time-series forecasting uses the continuations of historical segments similar to the current context as references for a forecaster...
arXiv:2609.35879v1 Announce Type: cross Abstract: Benchmarks for large language models (LLMs) typically evaluate the accuracy of answers against fixed reference labels. But a central step in many com...