Hugging Face Trending Papers
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 16
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
The paper introduces Conditional Program (CP), a compressed representation for large language model memory tables that reconstructs only the rows requested by an input. CP encodes each row with a short two‑bit code and a local scale, using a shared reconstruction program based on a Gaussian trellis. Experiments on three n‑gram memory families show CP reduces output KL divergence by 41–63% compared to 2‑bit round‑to‑nearest, and for DeepSeek it cuts table storage from 203 GB to 51 GB, enabling larger workloads on GPU.
arXiv Computation and Language
1d ago
Low-Bit N-Gram Embeddings for Language Models
The paper investigates reducing memory usage of n‑gram embedding tables in open‑weight language models by storing them in low precision. Experiments on nanochat, Llama, and GPT‑2 families show that four‑bit tables cut table memory by about 75% while retaining over 99% of the validation‑loss improvement. The study also demonstrates that re‑allocating the saved memory to larger tables can surpass the performance of full‑precision tables, and that similar savings are achievable in large models like DeepSeek‑V4.1‑Flash and Qwen3.8‑Flash‑Next with minimal impact on perplexity and downstream tasks.
arXiv Machine Learning
Oct 15
The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation
The paper introduces COUNT, a method that builds n‑gram embeddings without adding learned parameters by storing precomputed n‑gram counts on host memory. By weighting the language model’s own unembeddings with corpus frequencies, COUNT can be added as an extra input embedding during pretraining, improving validation loss equivalent to a 57% increase in compute for a 1B‑parameter model. Additionally, COUNT enables efficient GPU memory usage and allows domain‑specific n‑gram tables to be swapped in at inference time for training‑free adaptation.
arXiv Machine Learning
Jun 8
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
arXiv AI
3d ago
E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks
arXiv:2610.27699v1 Announce Type: new Abstract: Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar...
arXiv Machine Learning
Jul 9
Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference
arXiv:2607. 07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading.