arXiv Machine Learning

Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance

arXiv Machine Learning

Aug 17

The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference

arXiv:2608. 13756v1 Announce Type: new Abstract: Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable.

By Teng-Ruei Chen

llms efficiency benchmarks

arXiv Machine Learning

Jun 19

The Correctness Illusion in LLM-Generated GPU Kernels

arXiv:2606. 20128v1 Announce Type: cross Abstract: Benchmarks for LLM-generated GPU kernels (KernelBench, TritonBench, GEAK) score correctness through fixed-shape, small-sample allclose-style checks.

By Dipankar Sarkar

llms benchmarks

arXiv AI

Aug 7

Runtime Observability for Heterogeneous Attention Memory

arXiv:2608. 05863v1 Announce Type: new Abstract: Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differently under compression.

By Fanzhe Wei, Li Liu, Ziyang Wang, Chenyu Wang

efficiency

arXiv Machine Learning

Aug 11

Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving

arXiv:2608. 08910v1 Announce Type: cross Abstract: PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales.

By Matteo Grella

llms benchmarks

arXiv AI

Jul 17

A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi

arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.

By A. C. Opus, J. Q. Lu

rag multimodal benchmarks

arXiv Machine Learning

Aug 7

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

arXiv:2608. 05944v1 Announce Type: cross Abstract: We report operational experience full-fine-tuning a 32.

By Seon Ho Kim, Ui Jeong Jeon, Su Hyeon Kim, Min Tae Hwang

fine-tuning efficiency

arXiv Machine Learning

Jun 17

Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent Large Language Model Systems

arXiv:2606. 17182v1 Announce Type: new Abstract: Multi-agent LLM systems share state through memory stores, vector indices, and tool registries.

By Sajjad Khan

llms agents

arXiv Machine Learning

Aug 13

What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model

arXiv:2608. 10986v1 Announce Type: cross Abstract: A growing class of methods probes a language model by feeding it its own output: self-consistency, iterated refinement, agentic loops.

By Nicol\'as Vera Z\'u\~niga

llms agents

arXiv Machine Learning

Aug 14

A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family

arXiv:2608. 12700v1 Announce Type: new Abstract: Systems that generate GPU kernels with language models report high correctness rates.

By Rishi Shah, Rishav Shrestha

llms benchmarks safety

arXiv Machine Learning

Jul 31

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.

By Jiwon Jang, Kisu Yang, Heuiseok Lim, Hyunwoo Park

llms agents efficiency benchmarks safety

arXiv AI

Jun 10

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

arXiv:2606. 09864v1 Announce Type: cross Abstract: Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact.

By Bruce Changlong Xu, Adarsh Kumarappan, Mu Zhou

llms efficiency benchmarks safety

Hugging Face Trending Papers

Aug 6

Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening

We report operational experience full-fine-tuning a 32. 76B-parameter dense model (Qwen3-32B) on 16 x NVIDIA B300 (two nodes, FSDP / ZeRO-3) -- among the first published field accounts on this accelerator.

fine-tuning efficiency

Related stories