The paper presents the first instruction-level fault injection study of large language model (LLM) inference, focusing on how GPU soft errors affect these compute- and memory-intensive models. It examines the impact of model architecture, parameter scale, and task complexity on LLM vulnerability, revealing distinct reliability characteristics compared to earlier neural networks. The study offers new insights that can guide the development of more effective fault tolerance mechanisms for LLMs.
By Duo Chai, Zizhen Liu, Shuhuai Wang, Songwei Pei, Cheng Liu, Huawei Li, Shangguang Wang
arXiv:2606. 02430v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into high-performance computing (HPC) workflows, accelerating scientific discovery through diverse perspectives such as code generation and domain-specific decision-making.
By Yafan Huang, Sheng Di, Guanpeng Li
The study examines how reducing the width of deep neural networks through structured pruning affects their robustness to Single Event Upsets (SEUs). By iteratively pruning baseline models and conducting fault-injection tests at each stage, the authors find that while pruning increases per-inference sensitivity to bit-flips, the shorter execution time offsets this, lowering the overall SEU probability. Consequently, structured pruning can deliver energy and latency savings without compromising reliability for space-based AI systems.
By Toon Vinck, Na\"in Jonckers, Jaro De Roose, Jeffrey Prinzie, Peter Karsmakers
arXiv:2610.27621v1 Announce Type: cross
Abstract: Deep learning (DL) models are increasingly being deployed into radiation-challenging environments in spacecraft, particle accelerators, and, soon, sp...
By Peter Xiangyuan Ma, Joshua S. Bloom
arXiv:2603. 22770v2 Announce Type: replace-cross Abstract: The deployment of deep neural networks (DNNs) in safety-critical edge environments necessitates robustness against hardware-induced bit-flip errors.
By Alan T. L. Bacellar, Sathvik Chemudupati, Shashank Nag, Allison Seigler, Priscila M. V. Lima, Felipe M. G. Fran\c{c}a, Lizy K. John
The paper argues that large language models cannot achieve perfect reliability for any task, even with unlimited scale. It establishes that each generative task has an inherent reliability ceiling set by how much output uncertainty can be resolved from observable context, with a resolvable part that can be improved by more context and a subjective part tied to task ambiguity. The authors derive a scaling law showing that performance is limited by the scarcer resource—either training data or model capacity—and explain how this law explains phenomena such as retrieval augmentation and catastrophic forgetting.
By Subhabrata Majumdar
RobustNCL is a lightweight framework that enhances the reliability of neuromorphic continual learning (NCL) systems against transient hardware faults. It detects faults during weight access from off‑chip memory and cures affected weights using one of three strategies—zero replacement, min/max value, or most‑probable value—storing curing values in radiation‑hardened memory. Experiments on Split MNIST, Split Fashion‑MNIST, Split CIFAR‑10, and Split CIFAR‑100 show that RobustNCL maintains accuracy within 0.5% of fault‑free models and significantly outperforms baseline NCL methods while adding negligible latency and energy overhead.
By Rachmad Vidya Wicaksana Putra, Shuochen Zhou, Muhammad Shafique
The paper introduces Rondo, a stateful testing framework for deep‑learning compilers that models each test as a model plus an invocation sequence. Rondo mutates sequences using a Markov chain Monte Carlo sampler guided by compilation behavior, and employs a three‑way differential oracle to detect bugs that arise only when compiler state is retained. Evaluated on four major frameworks and 24 model architectures, Rondo uncovered 117 bugs—35 of which were stateful—leading to 39 bug reports, 35 confirmed by developers, and 18 fixes, while achieving high coverage and precision.
By Jiatao Ge, Ruixiang Qian, Yibo Hu, Yanzhou Mu, Yinglong Zou, Jiawei Liu, Zhenyu Chen, Chunrong Fang
arXiv:2607. 14112v1 Announce Type: cross Abstract: Large language models (LLMs) are evaluated as though perfect reliability is achievable for any task given sufficient scale.
By Subhabrata Majumdar
The paper investigates whether large language models (LLMs) can reinvent foundational computer‑science algorithms by suppressing direct recall of target algorithms through unlearning techniques. Experiments show that LLMs fail to recover nearly half of the algorithms, and even without unlearning, recovery rates drop dramatically, indicating an overreliance on memorized knowledge. The study introduces a generative verifier to maintain reasoning strength and concludes that current LLMs have limited capacity for foundational algorithm innovation.
By Jian Zhao, Haoren Luo, Yu Wang, Yuhan Cao, Pingyue Sheng, Tianxing He
arXiv:2608.30769v1 Announce Type: new
Abstract: LLM training is increasingly vulnerable to silent data corruption (SDC), yet existing protection methods largely treat Transformer computations uniform...
By Zhipeng Xia, Haotian Xu, Siyu Yun, Liqi Lin, Hu Liu, Yu Li, Cheng Zhuo
arXiv:2603. 11784v2 Announce Type: replace Abstract: As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed.
By Giorgio Racca, Michal Valko, Amartya Sanyal