arXiv Machine Learning July 14, 2026 By Weijia Han, Lisha Qu

Decomposing Runtime, Kernel, and Quantization Speedups via a Matched FP16 Intermediate: A Hardware-Conditioned Case Study on Four NVIDIA RTX A5000 GPUs

Read the original on arXiv Machine Learning →

arXiv:2607. 11368v1 Announce Type: cross Abstract: Reported serving speedups from quantized kernels typically bundle the weight format, the kernel, and the inference runtime into one number.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

efficiency

arXiv AI

Aug 12

Hand-Written PTX Tensor-Core GEMM Kernels: A Multi-Precision Study on NVIDIA L4

arXiv:2608. 10103v1 Announce Type: cross Abstract: High-performance Tensor Core kernels rely on a low-level PTX pipeline built from asynchronous data movement with cp.

By Matt J. Borowski, Blazej Osinski

efficiency

arXiv Machine Learning

Aug 7

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

arXiv:2608. 05944v1 Announce Type: cross Abstract: We report operational experience full-fine-tuning a 32.

By Seon Ho Kim, Ui Jeong Jeon, Su Hyeon Kim, Min Tae Hwang

fine-tuning efficiency

Hugging Face Trending Papers

Aug 6

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

We report operational experience full-fine-tuning a 32. 76B-parameter dense model (Qwen3-32B) on 16 x NVIDIA B300 (two nodes, FSDP / ZeRO-3) -- among the first published field accounts on this accelerator.

fine-tuning efficiency

arXiv AI

Jul 3

Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

arXiv:2607. 02043v1 Announce Type: cross Abstract: Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering.

By Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj, Renee St. Amant, Victor R\"uhle

llms benchmarks

arXiv Machine Learning

Aug 4

HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval

arXiv:2606. 21633v2 Announce Type: replace Abstract: The KV cache dominates GPU memory in long-context LLM serving, crowding out batch capacity and leaving GPU compute idle.

By Omin Kwon, Doyeon Kim, Jongseok Park, Seung Yul Lee, Ion Stoica, Jae W. Lee

llms diffusion efficiency benchmarks

arXiv AI

Jul 17

A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi

arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.

By A. C. Opus, J. Q. Lu

rag multimodal benchmarks

Related stories

Hand-Written PTX Tensor-Core GEMM Kernels: A Multi-Precision Study on NVIDIA L4

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval

A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi