arXiv AI

DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs

arXiv:2608. 14614v1 Announce Type: cross Abstract: As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets.

arXiv AI
Jul 3

The Rising Unsustainability of AI Graphics Cards Production

arXiv:2607. 01258v1 Announce Type: cross Abstract: The rapid advancement of Artificial Intelligence (AI) has been accompanied by significant increases in computational and environmental costs, driven by large-scale investments in AI infrastructure, hardware, and software.

By Cl\'ement Morand, Aur\'elie N\'ev\'eol, Anne-Laure Ligozat
arXiv AI
6d ago

Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware

The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.

By Philipp M. Z\"ahl, Elja Dalipaj, Anika Hennig, Timon Bayer
arXiv AI
Jun 16

The Energy Blind Spot: NVIDIA's Flagship Edge AI Hardware Cannot Support Process-Level Energy Attribution

arXiv:2605. 27599v2 Announce Type: replace-cross Abstract: Agentic AI workloads - where a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery - are being targeted for edge deployment, with NVIDIA, Dell, HP, ASUS, MSI, Acer, and Gigabyte all shipping GB10-based desktop AI systems in 2026.

By Deepak Panigrahy, Aakash Tyagi
arXiv Machine Learning
Jul 30

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

arXiv:2607. 26571v1 Announce Type: new Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems.

By Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis, George Dasoulas, Michael Keckeisen, Konstantinos Skianis, Sotirios Kotsopoulos, Francesca Dominici
arXiv AI
Jul 7

Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.

By Ferran Agullo, Joan Oliveras, Chen Wang, Alberto Gutierrez-Torre, Olivier Tardieu, Alaa Youssef, Jordi Torres, Josep Ll. Berral
Hugging Face Trending Papers
Jul 29

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting.

arXiv AI
Jun 9

Meeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKIT

arXiv:2601. 20408v2 Announce Type: replace-cross Abstract: Enterprise LLM deployment faces a critical scalability challenge: organizations must optimize models systematically to scale AI initiatives within constrained compute budgets, yet the specialized expertise required for manual optimization remains a niche and scarce skillset.

By Nicholas Santavas, Kareem Eissa, Patrycja Cieplicka, Piotr Florek, Matteo Nulli, Stefan Vasilev, Seyyed Hadi Hashemi, Antonios Gasteratos, Shahram Khadivi