Hugging Face Trending Papers
A Statistical Audit of Physical AI Benchmark Redundancy
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 1
A Statistical Audit of Physical AI Benchmark Redundancy
arXiv:2608.25940v2 Announce Type: replace-cross Abstract: Physical AI models are evaluated on suites of benchmarks that differ across model reports, leaving the model-by-benchmark matrix sparse and t...
arXiv AI
2d ago
Balance of Benchmarks: Semantic Density Reweighting for Task-Conditioned Model Comparison
The paper introduces Balance of Benchmarks (BoB), a framework that improves task-conditioned model comparison by weighting benchmark evidence based on semantic density, equating scores across varying difficulty levels, and pooling task-relevant residuals. BoB retains all eligible benchmark data while adjusting its influence, outperforming uniform averaging on the WildScores dataset with higher Spearman correlation, lower MAE, and better shortlist hit rates. The method also reduces ranking instability when benchmarks are repeated or paraphrased, and lowers retrospective regret in model selection.
arXiv AI
Jun 4
Position: State-of-the-Art Claims Require State-of-the-Art Evidence
arXiv:2605. 17273v3 Announce Type: replace-cross Abstract: State-of-the-Art (SOTA) claims pervade Artificial Intelligence (AI) and Machine Learning (ML) research.
arXiv AI
Jul 3
PACE: A Proxy for Agentic Capability Evaluation
arXiv:2607. 02032v1 Announce Type: new Abstract: Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure.
arXiv AI
Aug 20
Grouping the Stochastic Machine: Precision, Not Capability, as the Frontier Metric for AI Systems
The article argues that for frontier language models, precision—how consistently outputs cluster around the target—should be the key metric rather than capability, which measures average performance. It proposes a simple, non‑circular method to quantify precision by repeatedly scoring deterministic tasks and computing outcome consistency, and demonstrates how this metric can guide decisions about model improvements. The study shows that precision can reveal whether failures are due to systemic misalignment or random noise, and that real‑world measurement is more valuable than rule‑based benchmarks.
arXiv AI
Jul 29
Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation
arXiv:2607. 25891v1 Announce Type: new Abstract: Evaluating AI agents in interactive environments is hindered by fragmented tasks, scaffolds, verifiers, and scoring rules.