arXiv:2608. 06916v1 Announce Type: new Abstract: Quantized Neural Networks~(QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices.
arXiv:2311. 17815v3 Announce Type: replace-cross Abstract: Given their increasing size and complexity, the need for efficient execution of deep neural networks has become increasingly pressing in the design of heterogeneous High-Performance Computing (HPC) and edge platforms, leading to a wide variety of proposals for specialized deep learning architectures and hardware accelerators.
arXiv:2606. 20869v2 Announce Type: replace-cross Abstract: We present a holistic methodology for artificial intelligence algorithm and accelerator co-design, co-search, and co-generation (A3C3), which jointly optimizes neural network architectures and their hardware implementations to address the inefficiencies of traditional top-down AI system design flows.
The paper introduces RAMP, a method for robust adaptive mixed‑precision quantization of vision models on edge CPUs. It evaluates 13 sensitivity metrics across four neural networks, finding that Jensen‑Shannon Divergence consistently identifies layers that can be safely quantized. Using K‑Means clustering on these metrics, RAMP achieves near‑lossless accuracy with an average 1.81× speed‑up, while cautioning against excluding low‑speed‑up layers that can fragment the computational graph.
arXiv:2607. 18101v1 Announce Type: new Abstract: On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks.
The paper presents a novel multi‑exit computational scheme for TinyML on an ultra‑low‑power GAP9 SoC, adding confidence‑based gating points to a MobileNetV2 CNN for ImageNet‑100. By allowing inference to stop early, the approach cuts average MAC operations by 41 % (from 313 MMAC to 185 MMAC), reduces inference time by 29 % (49 ms to 35 ms), and saves 24 % in energy (2.1 mJ to 1.6 mJ per frame) with only a ~1 % drop in accuracy. Compared to a state‑of‑the‑art adaptive CNN on the same hardware, the method more than doubles computational efficiency, raising MAC/cycle from 8.1 to 17.2.
The paper argues that AI deployment performance depends on interactions among compression, compiler transformations, and serving policies rather than just model architecture. It introduces a three‑layer taxonomy—model‑level techniques, compiler transformations, and system policies—and frames deployment as a constrained multi‑objective optimization problem over accuracy, latency, throughput, memory footprint, and energy. The authors propose an evidence protocol for comparable benchmarking and synthesize data from edge and data‑center platforms to show that cross‑layer interactions drive deployment outcomes, concluding with a constraint‑aware selection procedure and open research problems.
arXiv:2608. 10506v1 Announce Type: cross Abstract: Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resource-constrained GPU platforms.
Para-Pipe is a hierarchical mapping framework that integrates intra- and inter-stage operator parallelism within a pipelined architecture for machine‑learning computational graphs on heterogeneous System‑on‑Chip (SoC) platforms. By selectively fine‑tuning parallelism levels across pipeline stages, it navigates the trade‑off between throughput and latency, reducing inter‑processor communication overhead and improving energy efficiency. Evaluation on Amlogic and Black Sesame SoCs shows multiple Pareto‑optimal configurations, with throughput‑optimized setups achieving up to 11.0% better energy efficiency than purely pipelined strategies and 23.3% better than non‑pipelined parallel execution.
arXiv:2606. 29518v1 Announce Type: cross Abstract: With the widespread adoption of AI in various IoT scenarios such as smart sensing and processing, AI chips have become a common component at the edge.
arXiv:2608.21646v1 Announce Type: cross Abstract: TinyML systems are enabling machine learning (ML) inference at the edge. However, there is little quantitative analysis of such systems. This paper p...
FeatureFormer is a neural performance predictor that adds explicit node-wise encodings of FLOPs, parameter counts, and memory proxies to a gated graph attention architecture. It is designed to improve latency and energy prediction for neural networks on edge devices, addressing the limitation of existing GNN and transformer predictors that largely ignore node-level computational cost. The authors also introduce NNEQ, a large-scale energy consumption dataset, and show through extensive experiments that FeatureFormer achieves state‑of‑the‑art performance across both metrics, including challenging out‑of‑domain settings, while the encoding can broadly enhance existing predictors with negligible overhead.