Hugging Face Blog

The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator

arXiv AI
Aug 20

TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving

TokenPowerSandbox is an evidence‑gated workflow that uses a CPU‑resident projector, brief GPU probes, full‑workload verification, and tamper‑evident provenance to predict energy usage of large language model serving. In experiments on an NVIDIA H100 80GB running Qwen2.5‑7B‑Instruct with vLLM, the method achieved energy MAPE of 6.23% and 7.35% on blind holdout and no‑refit confirmations, with high Spearman rank correlations. A predeclared TTFT gate demonstrated that energy accuracy alone cannot guarantee latency, as it passed at concurrency four but abstained below that level.

By Chenxu Niu
arXiv AI
Jun 2

How Much Progress Has There Been in NVIDIA Datacenter GPUs?

arXiv:2601. 20115v3 Announce Type: replace-cross Abstract: As the role of modern Graphics Processing Units (GPUs) becomes increasingly essential for several computing tasks, analyzing their past and current progress is paramount for determining future constraints on scientific research.

By Emanuele Del Sozzo, Martin Fleming, Kenneth Flamm, Neil Thompson
arXiv AI
Sep 28

ENAS: An Efficient Hardware-Aware Neural Architecture Search Framework for TinyML on Resource-Constrained Microcontrollers

ENAS is a hardware‑aware neural architecture search framework tailored for TinyML on microcontrollers. It uses a static feasibility check, a cell‑based search space with various block types and skip connections, and a three‑stage hybrid search strategy (random → top‑K → mutation) with cross‑run caching. The framework runs efficiently without GPUs, achieving significant search‑time speedups and competitive accuracy on Visual Wake Words and Melanoma Cancer benchmarks across a range of microcontrollers.

By Mohd Moin Khan, Naman Srivastava, Pandarasamy Arjunan