The paper presents a Pareto atlas of LLM inference optimizations, mapping cost, quality, and latency trade‑offs for Qwen2.5‑7B‑Instruct on L4, A100, and H100 GPUs. Using 54 measured configurations and a calibrated simulator, it identifies 18 of 36 setups on the Pareto frontier, showing that combined methods outperform single ones. Quality tests reveal that AWQ 4bit and FP8 weights offer significant latency reductions while largely preserving accuracy, but naive FP8 KV caching fails to answer any questions correctly.
By Srikanta Datta Tumkur, Jay Iyer, Mehar Simhadri, Sai Pavan Kumar, Sai Kapil Kumar, Ramesh Nampelly
arXiv:2605. 09708v2 Announce Type: replace-cross Abstract: We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in $n$-body problems, multi-field Boltzmann, neighbor-list molecular dynamics, multi-kernel PDE, FFT).
By V\'ictor Gallego
The paper introduces two scaling laws that predict the performance of safety probes—tools used to monitor language models for harmful behavior—based on model size and the number of labeled examples. By creating six large safety datasets and training probes across multiple model families, the authors show that probe error decreases as a power law with model size and further improves with more examples, with the improvement rate increasing for larger models. These laws enable researchers to forecast probe accuracy and decide whether to invest in more data or a larger model to achieve a desired error threshold.
By Yuchen Wu (University of Washington), Jerry Wei (Anthropic)
arXiv:2606.22826v2 Announce Type: replace
Abstract: Evaluating LLMs across many model variants---quantized, fine-tuned, or deployment-specific---requires running large benchmarks repeatedly, a proces...
By Devleena Das, Rajeev Patwari, Vikram Kumar Bukka, Nithin Kumar Guggilla, Elliott Delaye, Ashish Sirasao
RACE-AIMC is a framework that selects a single analog in‑memory computing (AIMC) accelerator from a pool to meet a specified energy budget while providing a mathematically exact upper bound on its error rate. Offline, it evaluates each chip, chooses the best one, and computes the bound; online, only that chip runs and a lightweight check decides whether to accept its output or defer to a fallback. Simulations show the certified error stays below 10% (mean 7.83%) and the system achieves clean‑digital accuracy while reducing energy use by about 69% compared to running all chips.
By Osama Yousuf, Martin Lueker-Boden
The paper demonstrates that the number of candidates generated during test-time scaling of large language models does not fully capture the system cost. By comparing different generation schedules (e.g., one batched call versus multiple serial calls) while keeping the total candidate count fixed, the authors show that serial calls consume significantly more GPU energy and latency. The study suggests that reporting candidate count alone is insufficient; evaluations should also include generation schedule and GPU-level metrics.
By Mobina Kashaniyan, Ali Jannesari
arXiv:2606. 21641v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been proposed as hyperparameter-optimization (HPO) advisors that "warm-start" search from prior knowledge, proposing strong configurations in very few evaluations.
By Carson Rodrigues, Oysturn Vas, Isaiah Abner DCosta, Nithish Kumar Prabhakaran
arXiv:2609.26272v1 Announce Type: new
Abstract: Neural samplers are trained against an unnormalised target $\tilde\pi=e^{-E}$ with no samples from $\pi$, which leaves the practitioner with no way to...
By Jian Xu
The paper investigates how activation quantisation affects different computational aspects of language models. While perplexity remains a reliable overall metric, it fails to reveal that induction accuracy stays high while retrieval accuracy drops significantly. Experiments show that the damage depends on more than just error magnitude, and that rotating the basis can recover induction performance even at low bit‑rates.
By Anish Sathyanarayanan
arXiv:2608. 07914v1 Announce Type: new Abstract: Behavioral contamination detectors can return "no evidence" either because a benchmark is clean or because the audit has little power.
By Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma
arXiv:2608. 01041v1 Announce Type: cross Abstract: Machine-learning predictors estimate processor performance far faster than cycle-level simulation.
By Yanxin Zhang, Shayne Wadle, Yuxuan Xiong, Zheyu Fu, Trivikram Krishnamurthy, Karu Sankaralingam
arXiv:2610.09541v1 Announce Type: cross
Abstract: Signals that predict whether a chain-of-thought (CoT) trace is correct are compared by AUC, but deploying one requires a threshold with a guarantee....
By Arjun Balaji