The New and Fresh analytics in Inference Endpoints

Welcome to Inference Providers on the Hub 🔥

Categories of Inference-Time Scaling for Improved LLM Reasoning

And an Overview of Recent Inference-Scaling Papers

Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation

arXiv:2605. 31278v2 Announce Type: replace-cross Abstract: Reliable evaluation of agentic systems requires unbiased estimates with valid uncertainty, but standard practice navigates between costly human annotation and biased LLM-as-judge proxies.

More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It

arXiv:2608. 14420v1 Announce Type: new Abstract: Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time.

Statistical Early Stopping for Reasoning Models

arXiv:2602. 13935v2 Announce Type: replace Abstract: While LLMs have seen substantial improvement in reasoning capabilities, they also sometimes overthink, generating unnecessary reasoning steps, particularly under uncertainty, given ill-posed or ambiguous queries.

Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements

The paper introduces Debiased Inference with Multiple Imperfect Measurements (DMM), a framework that uses several error‑prone AI measurements to perform valid downstream statistical inference without requiring costly gold‑standard labels. By assuming conditional independence of the measurements given the true label and unit‑level features, DMM leverages CP decomposition and semiparametric theory to prove consistency and asymptotic normality of its estimator. Simulations demonstrate that DMM yields valid inference and can improve efficiency when additional imperfect measurements are available, and the authors provide diagnostics for the key independence assumption.

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.

Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL

arXiv:2608. 15389v1 Announce Type: new Abstract: LLM-based Text-to-SQL progress is reported across heterogeneous benchmarks, backbones, and inference protocols, making cross-system comparison fragile.

Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

arXiv:2603. 11799v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models.

Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

arXiv:2608. 04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute.

Valid Inference with Synthetic Data via Task Exchangeability

arXiv:2606. 13629v1 Announce Type: cross Abstract: There is a proliferation of work arguing for the use of synthetic data in scientific research.

LLM-as-a-Judge Version Shift: The Instrument Changed, Not the Standard

arXiv:2610.32986v1 Announce Type: cross Abstract: Automated research pipelines increasingly rely on fixed LLM judges to rank, filter, and revise proposals, yet these judges are routinely replaced by...