arXiv Machine Learning June 9, 2026 By Ovishake Sen, Venkata Nithin Kamineni, Daniel Lobo, Swarup Bhunia, Rickard Ewetz, Baibhab Chatterjee

Characterizing the Impact of NVFP4 Quantization for Low-Power Edge AI Deployment

Read the original on arXiv Machine Learning →

arXiv:2606. 06527v2 Announce Type: replace-cross Abstract: Energy-efficient neural-network inference at the edge requires reducing arithmetic cost, memory traffic, computation energy, and storage overhead while maintaining acceptable accuracy.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 8

Ablation Study of Block Size, Weight Precision, and Scale Precision in NVFP4 Inference for Low-Power Edge-Efficient Neural Networks

arXiv:2606. 06527v1 Announce Type: cross Abstract: Energy-efficient edge inference requires reducing arithmetic cost, memory traffic, and hardware overhead.

By Ovishake Sen, Venkata Nithin Kamineni, Daniel Lobo, Swarup Bhunia, Rickard Ewetz, Baibhab Chatterjee

benchmarks

arXiv Machine Learning

Jul 21

Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator

arXiv:2607. 18101v1 Announce Type: new Abstract: On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks.

By Mateusz Piechocki, Alessandro Capotondi, Marek Kraft

fine-tuning efficiency

arXiv AI

Jun 26

SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference

arXiv:2606. 26587v1 Announce Type: cross Abstract: Low-bit floating-point formats and semi-structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation compression remains challenging: activations contain input-dependent outliers that dominate block scales in FP4 quantization, and directly applying N:M sparsity masks discards moderate values, coupling sparsification loss with quantization error.

By Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Huaqing Zheng, Xindian Ma, Peng Zhang

llms diffusion efficiency multimodal

arXiv AI

Jun 3

MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

arXiv:2605. 24391v2 Announce Type: replace-cross Abstract: As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference.

By Dahoon Park, Jahyun Koo, Sangwoo Hwang, Jaeha Kung

efficiency

arXiv AI

Jun 9

ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization

arXiv:2606. 07618v1 Announce Type: cross Abstract: NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales.

By Li Lin, Xiaojun Wan

llms efficiency

arXiv Machine Learning

Aug 10

MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model

arXiv:2608. 06916v1 Announce Type: new Abstract: Quantized Neural Networks~(QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices.

By Zijun Jiang, Yangdi Lyu

efficiency

Related stories

Ablation Study of Block Size, Weight Precision, and Scale Precision in NVFP4 Inference for Low-Power Edge-Efficient Neural Networks

Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator

SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference

MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization

MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model