Lossless compression can reduce the storage and movement of model weights without changing their floating-point values, but repeated statistical analysis and code construction add computational overhe...
The paper introduces Losslessly Compressed Tensors (LCT), a GPU‑friendly framework that repeatedly encodes and decodes tensors during training. LCT uses Huffman coding with precomputed codebooks to exploit redundancy in floating‑point data, achieving up to a 1.455× compression ratio. Experiments on models such as NanoGPT and Qwen3‑4B show memory savings of 24.6% and 23.8% respectively, enabling larger context lengths with only modest increases in step time.
By Max Zhu, Bohang Sun, Pietro Lio, Jialin Yu
arXiv:2605. 01708v3 Announce Type: replace-cross Abstract: Contemporary systems serving large language models (LLMs) have adopted prefill-decode disaggregation to load-balance between the compute-bound prefill phase and the memory-bound decode phase.
By Yipin Guo, Siddharth Joshi
BitSparse introduces a GPU‑resident, lossless compression format for sparse ReLU and ReLU² activations in feed‑forward networks. By integrating custom Triton kernels into an autograd function, it compresses and reconstructs activation caches without loss, supporting BF16 and FP8 formats and offering optional sign‑bit packing to approach theoretical limits. The method reduces per‑token VRAM usage by about 30% for Modded‑NanoGPT and Nemotron‑H, with negligible overhead and a new preallocated buffer system that eliminates host synchronization during training.
By Max Zhu, Bohang Sun, Pietro Lio, Jialin Yu
TensorDex is a lossless, tensor‑centric storage system designed to reduce the storage footprint of large language models by performing delta compression at the tensor level. It decomposes models into individual tensors, uses compact fingerprints to predict which tensors can be compressed together, and incrementally clusters tensors to select effective reference bases as new models arrive. In experiments on 2,890 Hugging Face models, TensorDex cuts storage usage by 70.5%, outperforms the previous best system by 37%, and achieves compression and decompression speeds of 22.9 GB/s and 28.4 GB/s, respectively.
By Tingfeng Lan, Zirui Wang, Yunjia Zheng, Zhaoyuan Su, Juncheng Yang, Yue Cheng
arXiv:2501.13558v4 Announce Type: replace
Abstract: Splat-based radiance fields have rapidly diversified in primitive type, kernel support, and appearance parameterization. This diversity makes compr...
By Francesco Di Sario, Riccardo Renzulli, Marco Grangetto, Akihiro Sugimoto, Enzo Tartaglione