arXiv Computation and Language
Nov 4

RAP: Research Attention Prediction in Language Agents

The paper introduces Research Attention Prediction (RAP), a benchmark designed to evaluate large language model (LLM) agents’ ability to infer changes across entire research fields from limited evidence gathered during online scholarly searches. RAP is built on a local search environment comprising 356,357 dated arXiv papers and includes two evaluation sets: a Core set of 2,160 questions covering 54 AI/ML fields and an Extended set spanning 269 fields. Experiments show that while search improves agents’ recognition of recent changes, their forecasting of future shifts remains limited or even negative, indicating that LLM agents can detect some recent developments but struggle to reliably anticipate future research activity.

By Yingqian Wu, Jingcong Liang, Shengbin Yue, Siyuan Wang, Zhenfei Yin, Junchi Yu, Zhongyu Wei
arXiv AI
Oct 19

AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents

AlphaBet is a reinforcement learning framework that trains a single large language model to search for evidence, reason about it, and report probabilities for unresolved events across multiple pre-resolution dates. By scoring each dated forecast with a proper scoring rule, AlphaBet uses one resolved event to supervise decisions under varying information states, decomposing forecasting regret into evidence-acquisition and probability-reporting gaps. Experiments on GLM-4.5-Air show that AlphaBet improves Brier scores on Forecast-Dojo and FutureX, attributing gains to multi-date training and better information acquisition and probability reporting.

By Liqin Ye, Rongzhi Zhang, Xin Liu, Zheng Li, Yuan He, Qingyu Yin, Priyanka Nigam, Chao Zhang
arXiv Computation and Language
Nov 5

Cognitive Bias Signatures in Search-Equipped LLM Agents: A Stage-Resolved Analysis

The study investigates how cognitive biases influence large language model (LLM) agents that use search. By controlling search conditions across 10 models and 1,000 synthetic search scenarios, the authors examine how the order of evidence, prior work records, and initial hypotheses affect retrieval, document opening, and final answers. Findings indicate that initial evidence order during active search has a stronger impact on outcomes than when all evidence is presented at once, and that dead‑end histories reduce strategy switching beyond what context length alone would explain.

By Yanru Xiao, Yingwei Yu
arXiv Machine Learning
Oct 3

Do Your Own Research: Learning to Forecast by Learning to Search

The paper introduces an agentic forecasting environment built on 2,100+ resolved Polymarket questions, where a language model (Qwen3.5-35B-A3B) learns to gather evidence during rollout via web search, page reading, and financial time series, all filtered to avoid post‑cutoff leaks. Training with single‑epoch GRPO and a Brier‑score reward improves calibration by 30‑40% and reduces search attempts, while the trained policy outperforms four frontier models in evidence‑based forecasting, achieving lower soft‑Brier scores at roughly 5% of the inference cost. The authors release the environment, dataset, and per‑rollout records as a reusable harness for temporal forecasting agents.

By Yusuf Afifi, Artur Kiulian, Anton Polishko, Mykola Khandoga, Hamudi Naanaa, Alina Krasnobrizha
arXiv Machine Learning
Sep 25

BridgeMem: Causal Dyadic Transition Residuals for Temporal Knowledge Graph Forecasting

BridgeMem is a new method for temporal knowledge graph forecasting that focuses on pair‑specific transition evidence, adding a residual correction to the log scores of a frozen full‑vocabulary forecaster. It retrieves and encodes prior events between a query actor and candidate, converting them into a likelihood‑ratio correction via a support‑adaptive empirical‑Bayes reader. Across five benchmarks, BridgeMem outperforms nine baselines from 2021–2026, improving filtered MRR and Hits@{1,3,10} metrics by up to 0.0216.

By Zeyan Li, Libing Chen, Shengda Zhuo, Yin Tang, Jianfeng Xu