Hugging Face Trending Papers

Sharp Limits for Honest Uncertainty in Hard-Budget Repeated Evaluation

Read the original on Hugging Face Trending Papers →

Repeated evaluation can estimate a benchmark score accurately while still requiring replication to certify narrow uncertainty. We characterize that requirement on a fixed grid of $M$ tasks with $L$ binary paths per task under the hard budget $(M+t)K$, where each path costs at most $K$ responses or episodes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Oct 9

Evaluating Budgeted Context Projection with Unexecuted Companion Runs

The paper examines how budgeted context projection can reduce the length of individual requests while affecting whether an agent completes within its budget. By analyzing a recorded ReVerPi source‑reading campaign, the authors show that retaining all intervention boundaries reveals trade‑offs that are obscured when a capped first continuation prevents companion runs, and they quantify success differences across various task subsets. The study highlights the importance of accounting for every boundary, preserving unknown outcomes, and separating completion from resource use in adaptive evaluation campaigns.

By Guangzhe Zhang