← Back to all news
arXiv Machine Learning By Christoph Berganski, Felix Jentzsch, Marco Platzner

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

Read the original on arXiv Machine Learning →

The paper presents a method for compiling multi‑threshold operators that support quantized, non‑monotonic, and fine‑granular activation functions in custom DNN accelerators. It extends the multi‑threshold representation to multi‑dimensional thresholds, integrates algebraic conversion rules into an ONNX‑based compiler, and validates the approach on both monotonic and non‑monotonic functions using the FINN framework. The authors also characterize hardware costs compared to graph‑level decomposition and discuss potential use in quantization‑aware training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 14

Efficient AI Model Deployment Using Quantization Analysis Tool

The paper introduces the Quantization Analysis Tool, a system built on the ONNX framework that streamlines quantization workflows for deep learning models. It offers layer‑wise sensitivity analysis, visualizations of weight and activation distributions, and guidance for selecting precision levels to balance model size, latency, and accuracy. Experiments on various neural network architectures show that the tool improves quantized accuracy and overall deployment efficiency.

By Dwith Chenna, Kanishka Macherla
efficiency
More like this →
arXiv AI
Oct 12

DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

DualQuant introduces a dual‑scale, calibration‑free post‑training quantization framework that jointly optimizes weight‑reconstruction MSE and derives per‑channel activation rescalings, enabling effective 4‑bit weight and activation quantization without calibration data. The method employs a non‑differentiable optimization solved via variable splitting and block coordinate descent, supporting a wide range of formats including MX and NVFP4. Experiments on Llama and Qwen models show DualQuant matches or surpasses calibration‑based SmoothQuant and significantly outperforms other calibration‑free baselines, achieving up to 2.61× memory reduction and 2.46× throughput speedup on an RTX 4090.

By Mahsa Salmani, Sayeh Sharify, Hesham Mostafa
llmsefficiency
More like this →