Hugging Face Trending Papers June 9, 2026

Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs

Read the original on Hugging Face Trending Papers →

We study dense-to-sparse continual training as a way to construct channel-sparse large language models from dense checkpoints. Starting from a Qwen2.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jul 31

Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs

arXiv:2607. 27591v1 Announce Type: new Abstract: Feed-forward networks (FFNs) dominate memory traffic and computation in large language model (LLM) inference, making them a primary target for activation sparsification.

By Jinyi Liu, Wei Chen, Pengyu Chen, Xinyi Yuan, Minghe Bai, Guoquan Wu, Jun Wei

llms efficiency

arXiv AI

Jul 22

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.

By Chao Han, Haozhe Hu, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv AI

Aug 6

Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs

arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.

By Yu Luo, Bo Dong, Wenhua Cheng, Haihao Shen

llms efficiency

arXiv Machine Learning

Jul 24

Overcoming the Communication-Performance Tradeoff in LLM Pretraining

arXiv:2508. 15706v3 Announce Type: replace Abstract: Communication-efficient distributed training algorithms (e.

By Amir Sarfi, Benjamin Th\'erien, Joel Lidin, Eugene Belilovsky

llms efficiency

arXiv AI

Jun 4

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.

By Devleena Das, Rajeev Patwari, Elliott Delaye, Ashish Sirasao

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 21

LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models

arXiv:2607. 16339v1 Announce Type: new Abstract: Diffusion-based Large Language Models(DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation.

By Xingru Chen, Zelang Liang, Yongjia Ma, Jiqing Zhan, Shuling Yang, Lian Wen, Kun Zhan

llms rag diffusion efficiency

Related stories

Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs

Overcoming the Communication-Performance Tradeoff in LLM Pretraining

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models