arXiv:2607. 05457v1 Announce Type: cross Abstract: Deep neural networks often contain substantial hidden-state redundancy, but most compression methods operate directly on weights, neurons, or quantised representations without explicitly characterising the dynamical role of internal states.
arXiv:2608. 05499v1 Announce Type: cross Abstract: Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices.
arXiv:2606. 26002v1 Announce Type: new Abstract: We present HiReLC, a hierarchical ensemble-reinforcement learning framework for automated joint quantization and structured pruning of deep neural networks.
Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithms that are hard to apply across architectures.
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources.
The paper introduces Neural Spectral Capacity (NSC), a closed‑form metric derived from the singular‑value spectrum of weight matrices that can be computed solely from a network’s architectural specification. Unlike traditional measures such as #Params and #FLOPs, NSC captures architectural structure (depth, width, head, FFN allocations) and can be evaluated without instantiating the model, data, or gradients. Using a dynamic‑programming solver (NSC‑DP), the authors demonstrate that NSC can efficiently identify architectures that outperform existing training‑free proxies across Transformer and CNN families, and achieve state‑of‑the‑art results in tasks such as WikiText‑103 and commonsense reasoning with LLaMA‑7B. whyItMatters":"NSC provides a fast, architecture‑only proxy that outperforms conventional metrics and training‑free proxies, enabling more effective design and pruning of large models without costly training or data."
LILA (Latent-Informed Layer Analysis) introduces a calibration‑free method for structured pruning of large language models by scoring neuron importance using the Kolmogorov–Smirnov distance between singular value distributions of full and neuron‑ablated feed‑forward network weight matrices. The approach requires no training, calibration data, or auxiliary networks, and outperforms existing methods such as PruneNet and SliceGPT on LLaMA‑2‑7B and Phi‑2 at various sparsity levels. After a single epoch of LoRA fine‑tuning, LILA matches heavily calibrated baselines, and a Neural Tangent Kernel analysis provides theoretical support for its spectral importance criterion. Additionally, LILA can dynamically allocate sparsity budgets, achieving state‑of‑the‑art generative preservation and revealing architectural bottlenecks at higher compression.
arXiv:2608. 00859v1 Announce Type: new Abstract: Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients.
arXiv:2606. 01126v1 Announce Type: cross Abstract: Pruning is a process designed to reduce the number of weights in a large neural network.
arXiv:2606. 01402v1 Announce Type: cross Abstract: Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.
The paper introduces LA-ReduNet, a lightweight adaptive version of the ReduNet neural network that uses hyperspherical manifold learning and adaptive step sizes to reduce the number of layers needed for the maximal coding rate reduction (MCR$^2$) objective to stabilize. By refining the layer‑wise update rule, LA-ReduNet achieves comparable classification accuracy while requiring far fewer layers and significantly less parameter storage—about 1/29 of the unfolded ReduNet module under the tested settings.
arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.
The paper presents a PyTorch-based framework for designing and optimizing binarized neural networks, incorporating freezing and pruning mechanisms. It introduces a novel pruning method that uses a global weighting scheme to assess parameter importance across abstraction levels, achieving a 70% pruning rate on VGG11 without sacrificing accuracy—outperforming existing binarized pruning results of 41%. The framework facilitates rapid, reproducible evaluation and prototyping of state‑of‑the‑art binarized network techniques.