Mistral AI

Mistral partners with NVIDIA to accelerate open frontier models

arXiv AI
Jun 2

How Much Progress Has There Been in NVIDIA Datacenter GPUs?

arXiv:2601. 20115v3 Announce Type: replace-cross Abstract: As the role of modern Graphics Processing Units (GPUs) becomes increasingly essential for several computing tasks, analyzing their past and current progress is paramount for determining future constraints on scientific research.

By Emanuele Del Sozzo, Martin Fleming, Kenneth Flamm, Neil Thompson
arXiv AI
Sep 15

GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

The paper predicts single‑sequence llama.cpp throughput from GGUF metadata using roofline‑shaped predictors with quantization‑specific scale factors. Experiments on 318 measurements across 53 host‑file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080 show that an active‑parameter decode model achieves significantly lower mean absolute percentage errors compared to models that use total parameters. The study also finds that low‑bit model ladders alter runtime ordering and that GGUF structure improves predictions, though fitted efficiencies vary across systems.

By Xinyu Qiu, Chuhong Xu, Bo Su, Ziyao Chen, Ruiyang Xu, Shimeng Dai