BenchMIRT: What are LLM benchmarks actually measuring?

Judge Arena: Benchmarking LLMs as Evaluators

Understanding the 4 Main Approaches to LLM Evaluation (From Scratch)

Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples

What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend

arXiv:2608. 04714v1 Announce Type: cross Abstract: Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are considered non-influential and their names and versions are almost never disclosed.

Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard

Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utility

arXiv:2607. 14108v1 Announce Type: cross Abstract: This paper introduces tool efficiency, a new quantitative metric to evaluate the rate of useful tool calls in an LLM agent trajectory.

Llama 2 on Amazon SageMaker a Benchmark