arXiv:2507. 14171v3 Announce Type: replace-cross Abstract: Importance-based structured pruning overwhelmingly relies on filter magnitude.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2509. 14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities.
arXiv:2604.13287v2 Announce Type: replace Abstract: Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre...
arXiv:2608. 06901v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved remarkable generalization across diverse multimodal tasks through large-scale pre-training, yet their rapidly increasing computational and memory requirements pose significant challenges for deployment in constrained environments.
The paper introduces a Hybrid Quadratic Unconstrained Binary Optimization (QUBO) framework for structured neural network pruning that integrates task‑aware sensitivity metrics (first‑order Taylor and Weight‑Fisher) into the objective’s linear term and optionally uses activation similarity for quadratic interactions. It controls pruning cardinality via a binary search over a capacity incentive rather than an explicit penalty and further refines the pruning mask with a two‑stage QUBO–Tensor‑Train strategy that employs gradient‑free black‑box optimization. Experiments on SIDD image denoising with a Half‑UNet model demonstrate that this Hybrid QUBO outperforms Taylor and L1‑based QUBO baselines in PSNR and SSIM, while also revealing computational and deployment challenges of mask‑based pruning.
LILA (Latent-Informed Layer Analysis) introduces a calibration‑free method for structured pruning of large language models by scoring neuron importance using the Kolmogorov–Smirnov distance between singular value distributions of full and neuron‑ablated feed‑forward network weight matrices. The approach requires no training, calibration data, or auxiliary networks, and outperforms existing methods such as PruneNet and SliceGPT on LLaMA‑2‑7B and Phi‑2 at various sparsity levels. After a single epoch of LoRA fine‑tuning, LILA matches heavily calibrated baselines, and a Neural Tangent Kernel analysis provides theoretical support for its spectral importance criterion. Additionally, LILA can dynamically allocate sparsity budgets, achieving state‑of‑the‑art generative preservation and revealing architectural bottlenecks at higher compression.
arXiv:2311.04902v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) with billions of parameters are prime targets for network pruning, which removes some model weights without hurt...