arXiv Machine Learning
GDN Tree-Scan: Served Tree Verification for Recurrent-Hybrid Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 24
TreeWY: Speculative Verification for Gated DeltaNet Hybrids
TreeWY introduces a speculative verification method for Gated DeltaNet (GDN) hybrid models that eliminates the need for per-draft-state snapshots. By applying a tree‑structured WY transform to the gated delta rule, each draft node’s output is computed with a single triangular solve, and only the accepted state is reconstructed on commit. Benchmarks on Qwen3.5 35B and 397B show reduced memory pressure, higher throughput, and lower time‑to‑first‑token in memory‑bound scenarios, while enabling wider, higher‑acceptance draft trees.
arXiv Machine Learning
Aug 4
Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
arXiv:2608. 01651v1 Announce Type: cross Abstract: Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memory-bound.
arXiv Computation and Language
Sep 23
LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay
The paper introduces LatentPort, a method that allows a language model to transfer its live memory to another model without requiring the receiver to reread the context. Experiments on a Qwen3.5 4B-to-9B sibling pair show that adding a Gated DeltaNet (GDN) persistent-state package reduces negative log‑likelihood by 0.747 nats/token and improves performance across 64 PG19 documents. The study also demonstrates that direct recurrent and convolution reuse outperforms learned GDN maps, and a 434,176‑parameter correction further narrows the performance gap to the native 9B model.
arXiv AI
Sep 1
Tail-Replay: Escaping the Curse of Linear Attention in Prefix Caching for Hybrid LLMs
arXiv:2608.30310v1 Announce Type: cross Abstract: Hybrid large language models interleave full-attention layers with linear-attention layers to reduce the cost of long-context inference. This structu...
arXiv Machine Learning
Sep 1
Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding
arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
arXiv AI
Jul 22
Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
arXiv:2607. 18553v1 Announce Type: cross Abstract: Can a language model read the quality of ongoing computation, and can an external intervention turn that readout into better outcomes?