This paper considers a decision-maker tasked with identifying the $k$ best items from a large pool of candidates, such as job applicants, LLMs, award nominees, or new product ideas. The decision-maker...
arXiv:2609.38860v1 Announce Type: cross Abstract: Learning from human preferences is central to large language model (LLM) alignment, but human preference annotation is costly. Active preference lear...
arXiv:2606. 15623v1 Announce Type: cross Abstract: Pairwise comparison is the gold standard for subjective ranking tasks; however, exhaustive annotation requires a massive number of human comparisons ($O(n^2)$).
arXiv:2605. 04267v2 Announce Type: replace Abstract: Interactive multi-objective optimization systems face a budget allocation dilemma: one can spend resources on expensive objective evaluations or on eliciting decision-maker preferences that identify the relevant region of the Pareto set.
arXiv:2610.14028v1 Announce Type: cross Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...
arXiv:2607. 25136v1 Announce Type: new Abstract: Research on preference optimization often varies the training objective while holding the data fixed.
arXiv:2610.14996v1 Announce Type: new Abstract: Choosing a few robust algorithms or schedules can require evaluating every candidate under many adverse scenarios, even though an agent only needs to i...
The paper introduces ExTS, a tree‑search policy designed for budget‑constrained agentic search where evaluation and generation costs are high. ExTS treats expansion as a value‑of‑information decision, combining discriminative reward shaping, a stochastic virtual child, and quality‑conditioned branching to allocate budget more effectively. Experiments on prompt optimization, code generation, molecular structure elucidation, and agentic workflow optimization show ExTS matching or surpassing task‑specific baselines with an average gain of +5.5% using a single configuration, and the authors also present pilot‑run diagnostics to guide adaptation to different problem structures.
arXiv:2409.00696v4 Announce Type: replace-cross Abstract: Rating-based human evaluation has become an essential tool to accurately evaluate the impressive performance of large language models (LLMs)....
arXiv:2610.18108v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to predict which of two candidate answers a human evaluator would prefer, providing a sc...
arXiv:2609.33180v2 Announce Type: replace-cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies...
arXiv:2602.02219v3 Announce Type: replace Abstract: Large language models are widely employed as evaluators, a paradigm commonly referred to as LLM-as-a-judge. Prior research has predominantly examin...