arXiv:2606. 14150v1 Announce Type: new Abstract: Pruning promises a shortcut to strong small language models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
Sparse autoencoders (SAEs) are commonly used to interpret large language models, but their reliability after pruning is unclear. This study shows that pruning’s effect on an SAE is governed by perturbation energy, a covariance-weighted norm, and that magnitude pruning distorts the representation space by ignoring activation geometry. Activation-aware pruning methods such as Wanda and SparseGPT better preserve SAE behavior, and the authors find that middle layers are especially vulnerable, leading them to propose a layer‑wise sparsity allocation that reduces perplexity for a given sparsity level.
arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly gro...
arXiv:2502. 07780v4 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved significant success across various NLP tasks.
arXiv:2605. 18331v2 Announce Type: replace Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years.
arXiv:2609.06557v1 Announce Type: new Abstract: Large language models (LLMs) are often considered fragile under aggressive sparsification, and maintaining reliable performance typically requires stic...
arXiv:2507. 01900v3 Announce Type: replace-cross Abstract: Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency.