Recent work in machine learning has demonstrated that transformer-based language model agents can communicate state not only through text, but also directly in latent space via key-value (KV) caches....
Latent Compaction is a test‑time training method that learns a compact, document‑specific key–value cache for large‑context language models. It initializes cache entries using reconstruction importance, teacher attention, and key‑space diversity, then fits attention outputs and log‑normalizers to the full cache before jointly optimizing the entire cache via end‑to‑end distillation. Experiments on RULER, LongBench, and Phonebook Lookup show significant accuracy gains over budget‑matched compression baselines, achieving 96.7% accuracy at 32× compression on RULER and transferring well across model scales and families.
By Yingfa Chen, Xingyu Shen, Chaojun Xiao, Xu Han
arXiv:2605. 22863v2 Announce Type: replace Abstract: LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model's state and encode at the receiver model.
By Maximillian Rossi, Prajwal Raghunath, Eugene Wu
arXiv:2607. 16213v1 Announce Type: new Abstract: Large Language Models (LLMs) generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major bottleneck.
By Soumia Bouyahiaoui, Manel Kara laouar, Aicha Boutorh, Mohamed Hadj Ameur
The paper introduces XKV, a latent protocol that enables efficient communication between heterogeneous language models by translating a sharer's key‑value cache into a receiver's context. XKV overcomes limitations of prior methods by jointly pooling both caches, reconciling differing layer depths, and allowing each receiver position to retrieve its own residual in native KV geometry. Across 45 dataset‑model pairings, XKV outperforms previous protocols and text communication while using fewer parameters and achieving faster translation times.
By Jiyao Liu, Qi Zhang, Yaoyi Jia, Ziwen Kan, Song Wang
arXiv:2602. 10238v2 Announce Type: replace-cross Abstract: The growing size of Large Language Models (LLMs) makes efficient inference challenging, primarily due to the memory demands of the autoregressive Key-Value (KV) cache.
By Luca Moschella, Laura Manduchi, Ozan Sener