Hugging Face Blog

The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator

arXiv AI
1d ago

TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving

TokenPowerSandbox is an evidence‑gated workflow that uses a CPU‑resident projector, brief GPU probes, full‑workload verification, and tamper‑evident provenance to predict energy usage of large language model serving. In experiments on an NVIDIA H100 80GB running Qwen2.5‑7B‑Instruct with vLLM, the method achieved energy MAPE of 6.23% and 7.35% on blind holdout and no‑refit confirmations, with high Spearman rank correlations. A predeclared TTFT gate demonstrated that energy accuracy alone cannot guarantee latency, as it passed at concurrency four but abstained below that level.

By Chenxu Niu
arXiv AI
Jun 2

How Much Progress Has There Been in NVIDIA Datacenter GPUs?

arXiv:2601. 20115v3 Announce Type: replace-cross Abstract: As the role of modern Graphics Processing Units (GPUs) becomes increasingly essential for several computing tasks, analyzing their past and current progress is paramount for determining future constraints on scientific research.

By Emanuele Del Sozzo, Martin Fleming, Kenneth Flamm, Neil Thompson