arXiv AI
Whether or Where? Tail Calibration Signals Finite-Budget Reward Hacking in Best-of-N Selection
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 15
Tail Competition Explains Scaling in Best-of-N Sampling for Verifiable Problems
arXiv:2610.19362v1 Announce Type: cross Abstract: Best-of-N sampling is a simple inference-time scaling strategy for large language models, yet its behavior under incomplete reward models remains poo...
arXiv Machine Learning
Oct 19
Coverage Cliffs in Off-Policy Quantile Certification
The paper introduces TailCert, a method for certifying an upper tail quantile of a policy’s performance using logged contextual‑bandit data. It builds a PAC‑Bayes Bernstein lower‑confidence envelope over a threshold grid and all data‑dependent posteriors, returning an atom‑safe upper confidence bound on the selected posterior’s pooled p‑quantile. The authors provide finite‑sample guarantees, analyze sample complexity on a rare‑action class, and demonstrate the method’s effectiveness through simulations.
arXiv Statistics ML
5d ago
Confidence Is Not a Promise: A Reference Audit of Typed Decision Models
The paper evaluates how well typed decision models, which output probability distributions, actually provide reliable guarantees. By comparing open-weight Laya and Jev 1.13 across 190 evaluations, the study finds that point‑estimate thresholds frequently break promised accuracy, while a Beta lower‑bound rule and Learn‑then‑Test approach maintain guarantees more consistently. The work also explores calibration across languages, the impact of noise, data requirements for fine‑tuning, and the limitations of confidence in predicting outcomes or providing faithful explanations.
arXiv Machine Learning
Jun 16
When Does q-error Predict Plan Regret? Three Regimes of Cardinality-Estimation Error
arXiv:2606. 15600v1 Announce Type: cross Abstract: Cardinality-estimation (CE) research ranks estimators by q-error, yet it is well known that q-error is an imperfect proxy for query-plan quality.
arXiv AI
Aug 21
Truncate Bad, Upweight Good: BoN-Style Distillation via Rank-Based Classification
arXiv:2608. 19748v1 Announce Type: cross Abstract: Inference-time selection methods, such as Best-of-N, improve generation by sampling a pool of candidates and selecting the top-ranked completion according to a reward model.
arXiv Machine Learning
Aug 11
Evaluator Ensembles Under Reward Hacking: Covariance Geometry and Finite-Search Guarantees
arXiv:2608. 08002v1 Announce Type: new Abstract: Language-model judges and reward models enable scalable supervision, but finite optimization can exploit evaluator errors rather than improve response quality.