arXiv:2606. 06527v2 Announce Type: replace-cross Abstract: Energy-efficient neural-network inference at the edge requires reducing arithmetic cost, memory traffic, computation energy, and storage overhead while maintaining acceptable accuracy.
arXiv:2606. 06527v1 Announce Type: cross Abstract: Energy-efficient edge inference requires reducing arithmetic cost, memory traffic, and hardware overhead.
arXiv:2607. 18101v1 Announce Type: new Abstract: On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks.
arXiv:2606. 26587v1 Announce Type: cross Abstract: Low-bit floating-point formats and semi-structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation compression remains challenging: activations contain input-dependent outliers that dominate block scales in FP4 quantization, and directly applying N:M sparsity masks discards moderate values, coupling sparsification loss with quantization error.
arXiv:2605. 24391v2 Announce Type: replace-cross Abstract: As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference.
arXiv:2606. 07618v1 Announce Type: cross Abstract: NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales.
arXiv:2512.04705v3 Announce Type: replace-cross Abstract: The deployment of Early-Exiting Neural Networks (EENNs) on edge accelerators requires optimizing not only the network architecture but also i...
arXiv:2608. 06916v1 Announce Type: new Abstract: Quantized Neural Networks~(QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices.
The paper introduces RAMP, a method for robust adaptive mixed‑precision quantization of vision models on edge CPUs. It evaluates 13 sensitivity metrics across four neural networks, finding that Jensen‑Shannon Divergence consistently identifies layers that can be safely quantized. Using K‑Means clustering on these metrics, RAMP achieves near‑lossless accuracy with an average 1.81× speed‑up, while cautioning against excluding low‑speed‑up layers that can fragment the computational graph.
HBQ: Hierarchical Scaling Block Quantization with Hardware‑Efficiency‑Aware Design for Accurate LLM Inference proposes a new block‑quantization scheme that uses large blocks and low‑overhead significand scaling to balance hardware efficiency and accuracy. The authors demonstrate that larger blocks improve efficiency by amortizing dequantization and accumulation costs, while their SIG scaling compensates for the resulting accuracy loss. Experiments on a 28 nm ASIC accelerator show that HBQ achieves up to 4.6× higher area/energy efficiency than state‑of‑the‑art weight‑only quantization, with 1.5–3.0× speedup and 1.6–3.3× system energy reduction over existing BQ methods.
arXiv:2606. 19964v1 Announce Type: new Abstract: Tsetlin Machine (TM) is a logic-based machine learning approach that relies on simple bitwise operations and finite-state automata, which makes it attractive for edge AI deployments.
Deploying deep learning models on edge CPUs is bottlenecked by computational and memory constraints. Mixed-precision quantization promises to reduce inference latency while preserving accuracy. Howeve...
arXiv:2607. 15810v1 Announce Type: new Abstract: Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP8.