Learning Functional Subspaces for Neural Network Compression
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2609.40127v1 Announce Type: cross Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...
arXiv:2606. 02559v1 Announce Type: cross Abstract: Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules.
arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.
arXiv:2607.12550v3 Announce Type: replace-cross Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference: it grows with batch size, context length, and depth, a...
arXiv:2610.32009v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models reduce computation through sparse activation, but storing their expert parameters remains a major memory bottleneck. We...
COMPOT is a training‑free compression framework for transformer models that uses a small calibration dataset to estimate a sparse weight factorization. It employs orthogonal dictionaries, enabling closed‑form Procrustes updates and analytical single‑step sparse coding, which dramatically reduces optimization cost. The method also introduces a one‑shot dynamic allocation strategy to adaptively redistribute compression rates across layers, achieving a superior quality‑compression trade‑off compared to strong low‑rank and sparse baselines while remaining compatible with post‑training quantization.