arXiv:2605. 25054v2 Announce Type: replace-cross Abstract: Deploying deep neural networks on resource-constrained 6G edge devices demands aggressive compression with minimal accuracy loss.
arXiv:2608. 06916v1 Announce Type: new Abstract: Quantized Neural Networks~(QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices.
arXiv:2510.08999v2 Announce Type: replace-cross Abstract: Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight...
Deploying deep learning models on edge CPUs is bottlenecked by computational and memory constraints. Mixed-precision quantization promises to reduce inference latency while preserving accuracy. Howeve...
arXiv:2609.28358v1 Announce Type: cross Abstract: Microscaling quantization techniques are increasingly used to represent neural network parameters with 8 bits or fewer while preserving near-full pre...
The paper introduces RAMP, a method for robust adaptive mixed‑precision quantization of vision models on edge CPUs. It evaluates 13 sensitivity metrics across four neural networks, finding that Jensen‑Shannon Divergence consistently identifies layers that can be safely quantized. Using K‑Means clustering on these metrics, RAMP achieves near‑lossless accuracy with an average 1.81× speed‑up, while cautioning against excluding low‑speed‑up layers that can fragment the computational graph.
arXiv:2605. 26660v2 Announce Type: replace Abstract: Quantization is an effective approach to reduce the memory footprint and inference cost of large language models (LLMs), yet maintaining performance in the ultra-low-bit regime remains challenging.
The paper introduces GQ-FSL, a green quantized federated split learning framework designed for wireless edge networks. It uses stochastic quantization for both local training and wireless transmissions, allowing asymmetric precision between client and server submodels to balance device energy limits with global convergence. The authors develop energy models and a convergence bound for heterogeneous data, then formulate an optimization problem to set the DNN split point and precision levels, achieving lower energy consumption while meeting latency and accuracy targets.
The paper introduces a curiosity‑driven quantized Mixture‑of‑Experts framework that routes inputs based on Bayesian epistemic uncertainty across heterogeneous experts (BitNet ternary, 1‑16 bit BitLinear, post‑training quantization). On audio classification benchmarks, 4‑bit quantization preserves 99.9 % of full‑precision F1 while achieving 4× compression and 31 % energy savings, and curiosity‑driven routing further improves accuracy and reduces cross‑fold variance by up to 85 %. The routing is self‑organizing, allocating the most uncertain samples to the high‑precision expert, and the method demonstrates statistical parity with full precision across datasets.
arXiv:2606. 04063v1 Announce Type: cross Abstract: Deploying large language models (LLMs) is challenging due to their significant memory and computational requirements.
Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithms that are hard to apply across architectures.
Squeeze10-LLM is a staged mixed‑precision post‑training quantization framework that reduces 16‑bit LLM weights to an average of 1.6 bits per weight by assigning 80% of weights to 1 bit and 20% to 4 bits. It introduces Post‑Binarization Activation Robustness (PBAR), a weight significance metric that considers activation impact, and Full Information Activation Supervision (FIAS), a strategy that preserves activation information to limit error propagation. Experiments on LLaMA and LLaMA2 demonstrate that Squeeze10‑LLM achieves state‑of‑the‑art performance for sub‑2‑bit weight‑only quantization, raising average accuracy from 43% to 56% on six zero‑shot classification tasks.
arXiv:2607. 29659v1 Announce Type: new Abstract: Deploying state-of-the-art deep neural networks (DNNs) at the wireless edge is severely bottlenecked by the strict energy and resource constraints of mobile devices.