arXiv Machine Learning
Correct but Unselectable: The Hidden Interface Tax in Multi-Candidate Reasoning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 21
Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration
arXiv:2607. 17531v1 Announce Type: cross Abstract: Test-time collaboration, including self-consistency, best-of-N selection, critic models, and verifier pipelines, is often credited with broadly improving LLM reasoning, yet its gains are uneven and sometimes negative.
arXiv Machine Learning
Oct 22
Nominate-then-Adjudicate: LLM-Assisted One-Pass and Per-Instance MIP Solver Tuning
The paper introduces a "nominate-then-adjudicate" pipeline for tuning mixed‑integer programming (MIP) solvers, where a fixed portfolio of configurations is sampled once and reused across instances, eliminating per‑instance search at inference. A 20‑feature representation maps structurally different instances to the same vector, enabling a one‑pass, zero‑solve selection process that outperforms default settings and per‑instance baselines such as ISAC+. The approach demonstrates significant practical gains, with the adjudicator achieving 538 wins over the top nominee and improving win rates against defaults from 84.9% to 90.2%.
arXiv AI
Sep 30
More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses
The paper introduces a controlled evaluation to disentangle answer coverage, repeatable task advantages, and gains from pre‑execution selection in large‑language‑model (LLM) harnesses. On 386 MATH‑500 tasks, eight generated harnesses and a baseline with nine identical copies were compared over three executions each, revealing that identical programs provide a 2.16‑point repeat‑averaged oracle headroom while generated programs show more repeatable score patterns but mainly expose persistent weaknesses. The study concludes that coverage and repeatability alone cannot justify claims of useful specialization and proposes an evaluation standard for harness diversity that requires task advantages to persist across executions and improve on additional fixed‑program executions under matched inference budgets.
arXiv AI
Oct 2
ReSolve: Reusing Candidate Reasoning through Selective Generative Moderation
ReSolve is a training‑free inference method that reuses candidate reasoning by selectively moderating generative outputs. It examines existing derivations when candidates disagree or lack a parseable answer, then incorporates new solutions into a bounded loop. On 130 competition‑mathematics problems, ReSolve achieves 100 and 99 correct answers with significantly fewer tokens than eight‑sample self‑consistency, while a controlled ablation shows that visible derivations improve accuracy.
arXiv Machine Learning
Oct 5
Online Verification of Language Model Responses Under Cost Constraints
The paper introduces OMVV, an online multi-verifier algorithm that maintains a pool of weak verifiers with varying costs and performance. It adaptively selects a verifier each round using an online score combiner and exponential-weights routing, providing distribution-free guarantees on false-accept and false-reject rates. Experiments on reasoning benchmarks show OMVV achieves higher accuracy at lower verification cost than any single fixed verifier across different budgets.
arXiv AI
Oct 23
ORSMT: Unleashing the Potential of Satisfiability Modulo Theories for Semantic Verification and Repair in LLM-Based Optimization
ORSMT is a tool that uses Satisfiability Modulo Theories to verify and repair artifacts produced by large language models when automating optimization modeling from natural language. It provides a typed semantic interface, exact SMT relation queries, and mechanisms to align decisions, project auxiliary variables, and reconstruct artifacts, thereby improving model-code match rates across thousands of problems. The system demonstrates significant gains in correctness and matching rates compared to existing methods such as ReLoop and Opt‑Verifier.