arXiv Computation and Language By Simon P. Villani

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

Read the original on arXiv Computation and Language →

The paper introduces LatentPort, a method that allows a language model to transfer its live memory to another model without requiring the receiver to reread the context. Experiments on a Qwen3.5 4B-to-9B sibling pair show that adding a Gated DeltaNet (GDN) persistent-state package reduces negative log‑likelihood by 0.747 nats/token and improves performance across 64 PG19 documents. The study also demonstrates that direct recurrent and convolution reuse outperforms learned GDN maps, and a 434,176‑parameter correction further narrows the performance gap to the native 9B model.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Hugging Face Trending Papers
Nov 9

Translating KV Caches Between Hybrid-Attention Language Models: MLA Latents, Linear-Attention States and Sparse-Attention Indexers

The paper presents a method for translating key‑value (KV) caches between hybrid‑attention language models, enabling a model to continue a conversation without re‑reading the entire history. It extends prior work (KV‑Lingo) to handle grouped‑query attention, compressed MLA latents, sparse‑attention indexer keys, and fixed‑size linear‑attention states, providing capture‑and‑inject points and mechanisms that preserve native quality. Experiments on 16–48 B hybrid pairs show near‑native performance and significant speedups, while also demonstrating effective long‑range recall and transfer to sparse‑attention targets.

arXiv AI
Oct 15

History Replays Itself: Unleashing Long-Context Capability in Linear Attention

arXiv:2610.15715v1 Announce Type: new Abstract: Linear attention promises the efficiency needed for long-context language modelling, yet its fixed-size recurrent state must simultaneously support ong...

By Li Nanbo, Firas Laakom, Yilan Zhang, Piotr Pi\k{e}kos, Yimeng Chen, Liangyu Wang, Yifan Xu, Mateusz Ostaszewski, Dandan Guo, Wenyi Wang, Yanning Dai, Yuhui Wang, J\"{u}rgen Schmidhuber