Hugging Face Trending Papers
NeuralZip: Reusable Setup for Fast Lossless Compression
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 8
NeuralZip: Reusable Setup for Fast Lossless Compression
NeuralZip is a method for lossless compression of floating‑point model checkpoints that pre‑computes a reusable setup based on exponent statistics. By grouping tensor chunks with similar exponent distributions, sharing Huffman codes, and packing recurring exponent tuples, it achieves 1.81–21.33× faster compression than baselines while preserving exact bit‑to‑bit reconstruction. The setup can be transferred across compatible architectures and reused during training, reducing GPU memory usage by up to 27.5% without affecting model outputs.
arXiv Machine Learning
Oct 16
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
The paper introduces Losslessly Compressed Tensors (LCT), a GPU‑friendly framework that repeatedly encodes and decodes tensors during training. LCT uses Huffman coding with precomputed codebooks to exploit redundancy in floating‑point data, achieving up to a 1.455× compression ratio. Experiments on models such as NanoGPT and Qwen3‑4B show memory savings of 24.6% and 23.8% respectively, enabling larger context lengths with only modest increases in step time.
arXiv Machine Learning
Jun 25
SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
arXiv:2605. 01708v3 Announce Type: replace-cross Abstract: Contemporary systems serving large language models (LLMs) have adopted prefill-decode disaggregation to load-balance between the compute-bound prefill phase and the memory-bound decode phase.
arXiv Machine Learning
Oct 16
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
BitSparse introduces a GPU‑resident, lossless compression format for sparse ReLU and ReLU² activations in feed‑forward networks. By integrating custom Triton kernels into an autograd function, it compresses and reconstructs activation caches without loss, supporting BF16 and FP8 formats and offering optional sign‑bit packing to approach theoretical limits. The method reduces per‑token VRAM usage by about 30% for Modded‑NanoGPT and Nemotron‑H, with negligible overhead and a new preallocated buffer system that eliminates host synchronization during training.
arXiv AI
Oct 16
TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models
TensorDex is a lossless, tensor‑centric storage system designed to reduce the storage footprint of large language models by performing delta compression at the tensor level. It decomposes models into individual tensors, uses compact fingerprints to predict which tensors can be compressed together, and incrementally clusters tensors to select effective reference bases as new models arrive. In experiments on 2,890 Hugging Face models, TensorDex cuts storage usage by 70.5%, outperforms the previous best system by 37%, and achieves compression and decompression speeds of 22.9 GB/s and 28.4 GB/s, respectively.
arXiv AI
Oct 5
Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
The paper introduces a hardware-software co‑design framework that compresses Mixture‑of‑Experts (MoE) model weights into low‑precision, hardware‑native sparse representations, enabling efficient execution on Sparse Tensor Cores (SpTCs). By relaxing discrete support selection through continuous reparameterization, the method jointly optimizes quantized weights and supports a router‑weighted reconstruction objective, achieving up to 4.35 percentage‑point gains in joint sparse‑quantization accuracy while retaining 96.09% of the original model’s performance. A custom grouped sparse GEMM kernel further boosts inference speed, outperforming NVIDIA’s baseline by up to 1.65× and reducing latency by up to 4.03× on B200 GPUs.