arXiv AI By Mahesh Ramesh, Eric Cho, Alice Lu, Shawn Huang, Baiqing Lyu

Hedge-Bench: Can Agents Reason Their Way to Better Predictions?

Read the original on arXiv AI →

Hedge-Bench is a benchmark comprising 92 real-world environments derived from hedge fund analysts' reasoning traces, designed to test AI agents’ ability to forecast future events from incomplete information. Agents are evaluated on joint success—correct forecasts with over 60% research coverage—across 11 frontier models, achieving rates between 2.54% and 31.52%. Training Muse‑Glimmer with GRPO on 165 environments improves coverage on held‑out tasks and demonstrates generalization to out‑of‑distribution evaluations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
3d ago

Benchmarking Forecasting Agents on Yesterday's Tomorrows

The paper introduces FABench, a scalable backtesting environment designed to evaluate forecasting agents. FABench automatically discovers events, generates questions, and resolves outcomes for any past forecast date, allowing continuous renewal beyond model training cutoffs. In initial tests, retrieval improves Brier scores across eight models, with top agents matching or surpassing market consensus on 54 Polymarket-linked questions, yet analysis shows persistent evidence‑weighting failures that highlight room for improvement.

By Yuxuan Wang, Yishu Wang, Jiaqi Deng
arXiv Computation and Language
1d ago

FinForesight: A Diagnostic Backtesting Benchmark for Deep Research Systems in Financial Forecasting

FinForesight is a diagnostic backtesting benchmark designed to evaluate deep research systems for financial forecasting. It tests four forecasting tasks—valuation, return ranking, event direction, and future financial fundamentals—across Chinese, U.S., and Hong Kong markets, while also including two diagnostic tasks for counterfactual reasoning and financial calculation. The framework limits evidence to what was available at the forecast date and compares pretraining-era, post-cutoff, and live evaluations to assess how well controlled backtests predict future performance.

By Fengyuan Liu, Mukai Li, Jieshun You, Yilin Zheng, Chenjia Min, Yizhou Zhang, Harper Chen, Qi Liu
arXiv Machine Learning
Oct 3

Do Your Own Research: Learning to Forecast by Learning to Search

The paper introduces an agentic forecasting environment built on 2,100+ resolved Polymarket questions, where a language model (Qwen3.5-35B-A3B) learns to gather evidence during rollout via web search, page reading, and financial time series, all filtered to avoid post‑cutoff leaks. Training with single‑epoch GRPO and a Brier‑score reward improves calibration by 30‑40% and reduces search attempts, while the trained policy outperforms four frontier models in evidence‑based forecasting, achieving lower soft‑Brier scores at roughly 5% of the inference cost. The authors release the environment, dataset, and per‑rollout records as a reusable harness for temporal forecasting agents.

By Yusuf Afifi, Artur Kiulian, Anton Polishko, Mykola Khandoga, Hamudi Naanaa, Alina Krasnobrizha