NeuralZip is a method for lossless compression of floating‑point model checkpoints that pre‑computes a reusable setup based on exponent statistics. By grouping tensor chunks with similar exponent distributions, sharing Huffman codes, and packing recurring exponent tuples, it achieves 1.81–21.33× faster compression than baselines while preserving exact bit‑to‑bit reconstruction. The setup can be transferred across compatible architectures and reused during training, reducing GPU memory usage by up to 27.5% without affecting model outputs.
By Mart\'in Bravo, Samuel Horv\'ath, Gonzalo Navarro, Andr\'es Abeliuk
Lossless compression can reduce the storage and movement of model weights without changing their floating-point values, but repeated statistical analysis and code construction add computational overhe...
arXiv:2605. 01708v3 Announce Type: replace-cross Abstract: Contemporary systems serving large language models (LLMs) have adopted prefill-decode disaggregation to load-balance between the compute-bound prefill phase and the memory-bound decode phase.
By Yipin Guo, Siddharth Joshi
BitSparse introduces a GPU‑resident, lossless compression format for sparse ReLU and ReLU² activations in feed‑forward networks. By integrating custom Triton kernels into an autograd function, it compresses and reconstructs activation caches without loss, supporting BF16 and FP8 formats and offering optional sign‑bit packing to approach theoretical limits. The method reduces per‑token VRAM usage by about 30% for Modded‑NanoGPT and Nemotron‑H, with negligible overhead and a new preallocated buffer system that eliminates host synchronization during training.
By Max Zhu, Bohang Sun, Pietro Lio, Jialin Yu
TensorDex is a lossless, tensor‑centric storage system designed to reduce the storage footprint of large language models by performing delta compression at the tensor level. It decomposes models into individual tensors, uses compact fingerprints to predict which tensors can be compressed together, and incrementally clusters tensors to select effective reference bases as new models arrive. In experiments on 2,890 Hugging Face models, TensorDex cuts storage usage by 70.5%, outperforms the previous best system by 37%, and achieves compression and decompression speeds of 22.9 GB/s and 28.4 GB/s, respectively.
By Tingfeng Lan, Zirui Wang, Yunjia Zheng, Zhaoyuan Su, Juncheng Yang, Yue Cheng
arXiv:2607. 16241v1 Announce Type: cross Abstract: Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench.
By Yunxiang Zhang (Xiangjun), Ping Yu (Xiangjun), Jianyu Wang (Xiangjun), Max (Xiangjun), Fan, Julian Reed, Azalia Mirhoseini, Will Su