The paper introduces DAC-Loop, a Distillation-Aware Compression Loop that combines knowledge distillation with structured channel pruning and unstructured sparsification to compress remote sensing CNNs. By iteratively applying pruning and distillation, the method achieves a better accuracy‑compression trade‑off than traditional one‑shot pruning or single‑stage knowledge distillation, maintaining strong performance even after reducing more than 90% of the model parameters on EuroSAT and NWPU‑RESISC45 datasets.
arXiv:2607. 06982v1 Announce Type: cross Abstract: Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks.
Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of CNNs hinders the deployment of CNNs onto resource-constrained embedded devices.
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources.
arXiv:2604.13287v2 Announce Type: replace Abstract: Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre...
arXiv:2606. 14886v1 Announce Type: cross Abstract: In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task.
arXiv:2607. 06915v1 Announce Type: cross Abstract: Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI.
Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.
arXiv:2609.13024v1 Announce Type: new Abstract: As a key model compression technique, knowledge distillation aims to transfer knowledge from a high-capacity teacher model to a lightweight student mod...
arXiv:2606. 15346v1 Announce Type: cross Abstract: Spatio-temporal prediction supports radar/satellite nowcasting and city-scale traffic monitoring, but modern models are often too expensive for real-time deployment.
arXiv:2610.15092v1 Announce Type: cross Abstract: Quantization plays a crucial role in efficiently deploying deep learning models on resource-constrained devices. Post-training quantization does not...
The paper introduces a lightweight joint source-channel coding framework for wireless image transmission that selectively replaces standard convolutional layers with depthwise separable convolutional layers. By varying the replacement ratio and position, the authors generate models with different computational complexities and evaluate how these changes affect reconstruction performance. Experiments show that replacing intermediate encoder and decoder layers yields a favorable complexity‑performance trade‑off, revealing redundancy in deep learning‑based JSCC systems and enabling substantial parameter reduction with only slight performance loss.
arXiv:2508. 13836v2 Announce Type: replace-cross Abstract: Pruning is a core technique for compressing neural networks to improve computational efficiency.