The paper introduces REQAP, a reliability‑aware quantized weight packing technique for systolic‑array DNN accelerators. It uses a sensitivity‑driven mixed‑precision quantization to assign layer‑wise bit‑widths, a deterministic register‑level packing strategy for SIMD‑within‑a‑register execution, and selective bit‑level protection that replicates critical MSBs into unused register space. Experiments on AlexNet, VGG‑11, and ResNet‑18 show up to 62% memory reduction, 56% fewer MAC operations, and improved accuracy resilience under fault injection compared to baseline and fully protected models.
By Mahdi Taheri, Samira Nazari, Mubassher Ansari, Ali Azarpeyvand, Mohsen Afsharchi, Maksim Jenihhin, Christian Herglotz
arXiv:2606. 09864v1 Announce Type: cross Abstract: Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact.
By Bruce Changlong Xu, Adarsh Kumarappan, Mu Zhou
arXiv:2603. 22770v2 Announce Type: replace-cross Abstract: The deployment of deep neural networks (DNNs) in safety-critical edge environments necessitates robustness against hardware-induced bit-flip errors.
By Alan T. L. Bacellar, Sathvik Chemudupati, Shashank Nag, Allison Seigler, Priscila M. V. Lima, Felipe M. G. Fran\c{c}a, Lizy K. John
arXiv:2605. 24391v2 Announce Type: replace-cross Abstract: As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference.
By Dahoon Park, Jahyun Koo, Sangwoo Hwang, Jaeha Kung
arXiv:2609.37887v1 Announce Type: new
Abstract: Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds...
By Arian Eamaz, Mojtaba Soltanalian
The paper introduces RAMP, a method for robust adaptive mixed‑precision quantization of vision models on edge CPUs. It evaluates 13 sensitivity metrics across four neural networks, finding that Jensen‑Shannon Divergence consistently identifies layers that can be safely quantized. Using K‑Means clustering on these metrics, RAMP achieves near‑lossless accuracy with an average 1.81× speed‑up, while cautioning against excluding low‑speed‑up layers that can fragment the computational graph.
By David Poblaci\'on-Criado, Dario Garcia-Gasulla, Eduardo Quinones
The paper introduces Right In-Place (RiP) convolution, a memory‑efficient strategy that corrects and generalizes previous in‑place convolution formulations to arbitrary stride, dilation, padding, and rectangular kernels. RiP aligns each layer’s input and output within a shared workspace, enabling safe, row‑major access with minimal memory overhead. Experiments on 10,000 random layers and 84 layers from 25 architectures show no corruption, matching or improving on existing herringbone workspaces while reducing memory usage by up to 24.8% and lowering peak activation memory on Raspberry Pi Pico MCUs by 12.5–33.3% without affecting cycle counts.
By Opegbemi Matthias Busoye, Tolulope Matthew Busoye, Eghonghon-aye Eigbe
arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.
By A. C. Opus, J. Q. Lu
arXiv:2608. 08081v1 Announce Type: cross Abstract: Large mixture-of-experts (MoE) language models with 26--120 billion parameters exceed the memory capacity of consumer devices through three simultaneous pressures: resident weight matrices, key-value (KV) cache state that grows linearly with context, and dozens of expert sublayers that must be paged on demand.
By Anthony. Lui, Mohamed. Elsaied, N. P. Savani
arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.
By Hamish Ogilvy
Deploying deep learning models on edge CPUs is bottlenecked by computational and memory constraints. Mixed-precision quantization promises to reduce inference latency while preserving accuracy. Howeve...
arXiv:2604. 26968v2 Announce Type: replace-cross Abstract: Key-value (KV) cache memory management is the primary bottleneck limiting throughput and cost-efficiency in large-scale GPU inference serving.
By Sanjeev Rao Ganjihal