arXiv AI
6d ago
LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs
arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).
Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization
arXiv:2607. 17019v1 Announce Type: new Abstract: We study whether \sigreg -- LeJEPA's anti-collapse objective -- can reshape representations during standard autoregressive language-model pretraining, and when the resulting geometry helps \kv-cache quantization.
Related stories
arXiv Machine Learning
Jul 7
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.
arXiv Machine Learning
Aug 4
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
arXiv:2608. 01247v1 Announce Type: cross Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.
Hugging Face Trending Papers
Aug 2
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained.
arXiv Machine Learning
Jul 2
GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache
arXiv:2607. 01065v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory.
arXiv Machine Learning
Jun 8
Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws
arXiv:2606. 06888v1 Announce Type: new Abstract: Classical scaling laws for language model pretraining balance model size against training dataset size under a fixed compute budget, assuming abundant data and a single pass over the corpus.
arXiv Machine Learning
Jun 9
SpectrumKV: Per-Token Mixed-Precision KV Cache Transfer for Prefill-Decode Disaggregated LLM Serving
arXiv:2606. 08635v1 Announce Type: new Abstract: Prefill-decode (PD) disaggregation decouples prompt processing from token generation, but it also turns the key-value (KV) cache into a network payload.
arXiv Machine Learning
Jul 20
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
arXiv:2607. 15456v1 Announce Type: new Abstract: Looped, weight-tied Transformers reduce parameters by reusing a block, but decoding still stores a separate K/V cache for every recurrence step.
arXiv Machine Learning
Jul 15
A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs
arXiv:2607. 12550v1 Announce Type: new Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference.
arXiv Machine Learning
Aug 11
SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Coding
arXiv:2608. 07915v1 Announce Type: new Abstract: Large language models (LLMs) increasingly read long inputs in the agentic era, from whole documents and codebases to conversations across many turns.
arXiv AI
Jun 26
Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders
arXiv:2606. 27321v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have become a leading tool for interpreting the representations of vision foundation models, decomposing their polysemantic activations into a larger set of sparse, more monosemantic features.
arXiv Machine Learning
Jun 3
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
arXiv:2606. 03458v1 Announce Type: new Abstract: Test-time scaling is a powerful approach to obtain better reasoning in large language models, but it becomes memory-bottlenecked during long-horizon decoding, as the KV-cache grows.