Hugging Face Trending Papers

When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale

Read the original on Hugging Face Trending Papers →

The paper investigates how quantization and pruning affect language models of different sizes. It shows that compression damages specific components—particularly attention heads used for in‑context retrieval—more severely in larger models, and that the network’s ability to compensate for this damage diminishes as model size grows. Consequently, compression techniques that appear benign on small models can become costly on larger ones.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Nov 6

When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale

The paper investigates how quantization and pruning—common compression techniques—affect language models of different sizes. By isolating a small set of attention heads responsible for in-context retrieval, the authors decompose the loss from ablating these heads into components that are compensated by the rest of the network and components that directly impact the output. Their findings show that while smaller models largely recover from such damage, larger models retain most of the loss because their compensatory mechanisms are less effective, indicating that compression can be far more detrimental as models scale.

By Gautam Gupta, Dikshant Kukreja, Unique Kumar, Avni Mittal, Deepanshu Dabas, Kshitij Sah, Erik Cambria, Rajiv Ratn Shah, Zhengkui Wang, Avinash Anand