arXiv Machine Learning July 21, 2026 By Libo Sun, Po-Wei Harn, Zewei Zhang, Peixiong He, Xiao Qin

Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization

Read the original on arXiv Machine Learning →

arXiv:2607. 17019v1 Announce Type: new Abstract: We study whether \sigreg -- LeJEPA's anti-collapse objective -- can reshape representations during standard autoregressive language-model pretraining, and when the resulting geometry helps \kv-cache quantization.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

efficiency

arXiv AI

6d ago

LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).

By Yirui Liu, Ruoling Qi, Longwen Wang, Xuaner Wu, Jian Chen, Yuxin Jin, Jiawei Shao, Xuelong Li

llms efficiency

arXiv Machine Learning

Jul 7

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.

By Hamish Ogilvy

llms efficiency

arXiv Machine Learning

Aug 4

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

arXiv:2608. 01247v1 Announce Type: cross Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.

By Changwoo Baek, Seungjun Shin, Kyeongbo Kong

fine-tuning efficiency benchmarks

Hugging Face Trending Papers

Aug 2

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained.

fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 2

GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache

arXiv:2607. 01065v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory.

By Soosung Kim, Minjae Park, Eui-Young Chung, Jaeyong Chung

llms efficiency safety

arXiv Machine Learning

Jun 8

Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws

arXiv:2606. 06888v1 Announce Type: new Abstract: Classical scaling laws for language model pretraining balance model size against training dataset size under a fixed compute budget, assuming abundant data and a single pass over the corpus.

By Zhiwei Xu, Shihao Wu, Hanseul Cho, Wei Hu, Yixin Wang

llms diffusion

Related stories

LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache

Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws