arXiv:2609. 29140v1 Announce Type: new Abstract: Repeated evaluation can estimate a benchmark score accurately while still requiring replication to certify narrow uncertainty.
By Yezhou Cheng, Runjia Du, Zeming Liu, Qibai Chen, Hang Lyu, Yilan Wei, Yankai Zeng, Bojun Lin
Repeated-sampling evaluations increasingly extrapolate pass@k far beyond the number n of samples collected per problem. We show that, in the pooled/random-task conditional-Binomial model, fixed-n succ...
arXiv:2609.09245v1 Announce Type: new
Abstract: Repeated-sampling evaluations increasingly extrapolate pass@k far beyond the number n of samples collected per problem. We show that, in the pooled/ran...
By Pranav Singh, Prashant Singh
arXiv:2606. 29654v1 Announce Type: new Abstract: Multi-agent deliberation among LLMs can improve reasoning, but deployment requires deciding when the current answer is reliable enough to act on and when it should be escalated to human review.
By Mengdie Flora Wang, Haochen Xie, Guanghui Wang, Devin Zhang, Jae Oh Woo
arXiv:2606.00920v2 Announce Type: replace-cross
Abstract: A benchmark score is a measurement whose uncertainty has two sources: an aleatoric component from stochastic decoding and an epistemic compon...
By Yongxi Zhou, Lai Yun Choi, Jiaxi Wen, Wenbo Ye, Junwei Yao
The paper examines how budgeted context projection can reduce the length of individual requests while affecting whether an agent completes within its budget. By analyzing a recorded ReVerPi source‑reading campaign, the authors show that retaining all intervention boundaries reveals trade‑offs that are obscured when a capped first continuation prevents companion runs, and they quantify success differences across various task subsets. The study highlights the importance of accounting for every boundary, preserving unknown outcomes, and separating completion from resource use in adaptive evaluation campaigns.
By Guangzhe Zhang