← Back to all news
arXiv Machine Learning By Mrityunjay Shukla, Shubham Roy, Biswabandan Panda

Amplifying Latency and Cost of LLM Inference via Rowhammer Attack

Read the original on arXiv Machine Learning →

The paper introduces LLVMhammer, a Rowhammer attack that induces a bit‑flip in Triton’s MLIR/LLVM compilation pipeline, causing the generation of a suboptimal GPU kernel that persists across inference runs. A single successful bit‑flip can increase LLM inference latency by 4–16 % on NVIDIA RTX 4090 and AMD RX 7900 XTX GPUs, and can also degrade output quality, lowering BERTScore to as low as 25 %. The attack is GPU‑vendor‑agnostic and demonstrates that memory corruption in the compiler can have lasting performance and security impacts on large‑language‑model inference.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 23

Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

The paper addresses the problem of non‑deterministic outputs from large language models (LLMs) when run on different GPU architectures, caused by floating‑point non‑associativity and hardware‑dependent kernel choices. It proposes a set of fixed‑configuration fused‑upcast GEMM kernels that load 16‑bit weights, upcast to FP32, and perform IEEE‑754 compliant reductions in a problem‑shape‑dependent order, ensuring identical linear‑layer outputs across NVIDIA Ampere, Ada, and Hopper GPUs. The new approach achieves 1.17–3.1× faster end‑to‑end performance than existing solutions and halves weight‑memory traffic while maintaining cross‑architecture reproducibility.

By Liam Cooper, Shinnung Jeong, Hyeran Jeon, Jeffrey Young, Hyesoon Kim
llmsbenchmarks
More like this →