← Back to all news
arXiv AI By Jingjie Ning, Xueqi Li, Yibo Kong

SearchHorizonBench: A Benchmark for Search Value in AI Agents

Read the original on arXiv AI →

SearchHorizonBench is a new benchmark that evaluates how AI agents decide whether the cost of an additional search is justified by its expected benefit. It provides 45,632 executions on 3,168 questions, using paired continuations and repeated runs to separate true search gain from generation noise. The dataset shows that historical calibration can reduce Brier error by 16.2% and that free‑selection observations offer a 2.49‑point utility advantage over a constant action, highlighting the importance of accurate search‑value estimation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Oct 27

SearchJev: A Calibrated System-1 Model for Search Agents

arXiv:2610.05107v2 Announce Type: replace-cross Abstract: Search agents repeatedly make short decisions about relevance, evidence sufficiency, and search actions. Using generative language models for...

By Congfeng Cao, Lipeng Zuo, Konstantinos Papakostas, Qiwei Xu, Songwei Xu, Qianru Meng, Lun Zhou, Zhaochun Ren, Yougang Lyu, Xiaohui Yan
llmsagentsbenchmarks
More like this →
arXiv AI
3d ago

Heterogeneous Co-Evolution Learning for Tool-Efficient Search Agents with Zero Labels

The paper introduces ZEST-Search, a zero-label framework that trains search agents by co-evolving task generation and solver learning. It constructs validated search tasks online from unlabeled questions, using a heterogeneous proposer to adapt task difficulty to the solver’s evolving capabilities. The approach rewards the proposer for intermediate difficulty and the solver for tool-aware performance, leading to improved search efficiency and accuracy across multiple question‑answering benchmarks.

By Qin-Wen Luo, Sheng Ren, Xiang Chen, Rui Liu, Jun Fang, Naiqiang Tan, Jun Wang, Hua Chai, Sheng-Jun Huang
llmsagentsnlpreinforcement-learningbenchmarks
More like this →
arXiv Machine Learning
Sep 22

Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning

The paper introduces Strategy Accumulation and Guided Execution (SAGE), a two-stage framework that makes automated fine-tuning of large language models cumulative. In the first stage, a multi-agent pipeline uses Monte Carlo Tree Search to explore training strategies while a Distillation Agent records task-specific insights and cross-task confidence scores into a structured repository. In the second stage, SAGE retrieves relevant experience from this repository to guide training on new tasks, achieving a 12.4‑percentage‑point improvement over a baseline pipeline without accumulated experience on nine unseen tasks.

By Haoran Zhao, Wei Du, Dingwen Yang, Jixuan Huang, Junlin Shang, Lingyong Fang, Ya Guo, Tao Gui, Qi Zhang, Xuanjing Huang
llmsagentsfine-tuningefficiency
More like this →