arXiv:2607. 17531v1 Announce Type: cross Abstract: Test-time collaboration, including self-consistency, best-of-N selection, critic models, and verifier pipelines, is often credited with broadly improving LLM reasoning, yet its gains are uneven and sometimes negative.
The paper introduces a "nominate-then-adjudicate" pipeline for tuning mixed‑integer programming (MIP) solvers, where a fixed portfolio of configurations is sampled once and reused across instances, eliminating per‑instance search at inference. A 20‑feature representation maps structurally different instances to the same vector, enabling a one‑pass, zero‑solve selection process that outperforms default settings and per‑instance baselines such as ISAC+. The approach demonstrates significant practical gains, with the adjudicator achieving 538 wins over the top nominee and improving win rates against defaults from 84.9% to 90.2%.
The paper introduces a controlled evaluation to disentangle answer coverage, repeatable task advantages, and gains from pre‑execution selection in large‑language‑model (LLM) harnesses. On 386 MATH‑500 tasks, eight generated harnesses and a baseline with nine identical copies were compared over three executions each, revealing that identical programs provide a 2.16‑point repeat‑averaged oracle headroom while generated programs show more repeatable score patterns but mainly expose persistent weaknesses. The study concludes that coverage and repeatability alone cannot justify claims of useful specialization and proposes an evaluation standard for harness diversity that requires task advantages to persist across executions and improve on additional fixed‑program executions under matched inference budgets.
ReSolve is a training‑free inference method that reuses candidate reasoning by selectively moderating generative outputs. It examines existing derivations when candidates disagree or lack a parseable answer, then incorporates new solutions into a bounded loop. On 130 competition‑mathematics problems, ReSolve achieves 100 and 99 correct answers with significantly fewer tokens than eight‑sample self‑consistency, while a controlled ablation shows that visible derivations improve accuracy.
The paper introduces OMVV, an online multi-verifier algorithm that maintains a pool of weak verifiers with varying costs and performance. It adaptively selects a verifier each round using an online score combiner and exponential-weights routing, providing distribution-free guarantees on false-accept and false-reject rates. Experiments on reasoning benchmarks show OMVV achieves higher accuracy at lower verification cost than any single fixed verifier across different budgets.
ORSMT is a tool that uses Satisfiability Modulo Theories to verify and repair artifacts produced by large language models when automating optimization modeling from natural language. It provides a typed semantic interface, exact SMT relation queries, and mechanisms to align decisions, project auxiliary variables, and reconstruct artifacts, thereby improving model-code match rates across thousands of problems. The system demonstrates significant gains in correctness and matching rates compared to existing methods such as ReLoop and Opt‑Verifier.
arXiv:2606. 16999v1 Announce Type: cross Abstract: Frozen small code models ( =45.
arXiv:2609.39496v1 Announce Type: cross Abstract: Typed decision models such as Jev offer an efficient alternative to generative LLMs in decision-making workflows by selecting directly from predefine...
The paper argues that partition-level scores used in decomposed algorithm selection overestimate what a deployable selector can achieve. It introduces the deployment‑fidelity gap, G(R), which measures the difference between partition‑level and end‑to‑end utility, and derives conditions under which a partition choice is truly deployment‑optimal. Experiments on five public benchmarks show that every decomposed pipeline has a positive gap, with some cases where the partition advantage disappears or reverses when evaluated end‑to‑end, highlighting the need to report both partition and end‑to‑end scores.
The paper presents a unified evaluation of seven open reasoning language models across four benchmarks (ARC-Challenge, GSM8K, MATH levels 1–3, and TruthfulQA MC1) using a consistent 238-example subset and three prompting strategies (zero-shot, chain-of-thought, few-shot CoT). It reports not only accuracy but also Wilson confidence intervals, latency, VRAM usage, weighted aggregate performance, Pareto-efficient points, prompt-sensitivity, and compatibility diagnostics, revealing that Gemma-4-26B-A4B tops the weighted score while Gemma-4-E4B offers a strong practical trade-off. The study emphasizes that model rankings shift with prompting strategy and that deployment trade-offs remain crucial, advocating for a deployment-aware, multi-objective evaluation framework rather than a single-score leaderboard.
The paper investigates how to estimate confidence for activation oracles—language models that translate internal activations of other models into natural language. Five confidence estimation methods are compared across four oracles from Qwen and Gemma families (8B–27B parameters) on a secret‑word recovery task. Results show that when a hypothesis space is available, scoring candidate answers directly yields the best accuracy and confidence separation; when free‑text answers are required, sampling agreement over twenty outputs provides the most calibrated confidence, while a rescaled probability from a calibration set performs comparably with fewer generations. Asking the oracle to output a numeric confidence score proves ineffective.
arXiv:2606. 01246v1 Announce Type: new Abstract: Text-to-SQL on complex schemas is unreliable on a single pass, so recent systems generate multiple SQL candidates and let voting filter out errors.