arXiv Machine Learning
Aug 26

Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing

The paper introduces the Sparse-Activation-ReLU (SAR) layer, a single‑step neural operator that promotes activation sparsity without surrogate‑gradient training and is compatible with event‑based computing. In a trunk‑based NOMAD architecture, SAR improves the combined Latency‑Error‑Energy (LEE) metric by over fivefold compared to Variable Spiking Neuron (VSN) and Leaky Integrate‑and‑Fire (LIF) models. Additional techniques such as synthetic knowledge distillation, a ReLU‑based spiking loss, and graph‑neighbor thresholding further reduce LEE and L2 error on the Heat Exchanger dataset, advancing energy‑efficient virtual sensing for edge deployment.

By William Howes, Farid Ahmed, Syed Bahauddin Alam