arXiv AI August 6, 2026 By Shahed Masoudian, Passant Shafaei, Monorama Swain, Markus Schedl

What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend

arXiv:2608. 04714v1 Announce Type: cross Abstract: Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are considered non-influential and their names and versions are almost never disclosed.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 13

Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations

arXiv:2607. 09172v1 Announce Type: cross Abstract: Large Language Models are reshaping how software is developed and maintained.

By Nada Zine, Tristan Coignion, Vincenzo Stoico, Cl\'ement Quinton, Romain Rouvoy, Patricia Lago

llms

arXiv AI

Jun 26

The Capability Frontier: Benchmarks Miss 82% of Model Performance

arXiv:2606. 26836v1 Announce Type: new Abstract: Existing benchmarks typically report accuracy for a single model on a single run.

By Bradley Fowler, Ryan Smith, Daniel Thi Graviet, William Myers, Joshua Greaves, Narmeen Fatimah Oozeer, Ant\'ia Garc\'ia, Philip Quirke, Amirali Abdullah, Fazl Barez, Shriyash Kaustubh Upadhyay

llms agents benchmarks

arXiv AI

Jul 24

Profiling Lightweight Large Language Models

arXiv:2607. 20806v1 Announce Type: new Abstract: Lightweight large language models (LLMs) are increasingly being deployed locally on personal computers and are expected to play a growing role in resource-constrained edge and mobile environments.

By Tomohiro Harada, Enrique Alba, Gabriel Luque

llms benchmarks

arXiv AI

Jun 11

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.

By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu

llms benchmarks

arXiv AI

Aug 13

The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance

arXiv:2608. 11694v1 Announce Type: cross Abstract: A benchmark score comes from a single phrasing of each problem.

By Shailja Thakur, Sungeun An, Chad DeLuca, Hima Patel

llms benchmarks

arXiv AI

Aug 11

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

arXiv:2608. 09351v1 Announce Type: cross Abstract: Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment.

By Nikita Kozodoi, Zainab Afolabi, Jack Butler

llms nlp multimodal

One email a morning, machine-written

Related stories

Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations

The Capability Frontier: Benchmarks Miss 82% of Model Performance

Profiling Lightweight Large Language Models

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute