arXiv AI August 24, 2026 By Sneha Murthy Ghantasala

TreeWY: Speculative Verification for Gated DeltaNet Hybrids

TreeWY introduces a speculative verification method for Gated DeltaNet (GDN) hybrid models that eliminates the need for per-draft-state snapshots. By applying a tree‑structured WY transform to the gated delta rule, each draft node’s output is computed with a single triangular solve, and only the accepted state is reconstructed on commit. Benchmarks on Qwen3.5 35B and 397B show reduced memory pressure, higher throughput, and lower time‑to‑first‑token in memory‑bound scenarios, while enabling wider, higher‑acceptance draft trees.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 4

Bole: Efficient Tree Speculation for Hybrid-Attention Language Models

arXiv:2608. 01651v1 Announce Type: cross Abstract: Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memory-bound.

By Li Wang, Yi Su, Xiabao Wu, Chiran You, Yongchao Liu, Zhan Qiu, Juelu Zhang, Jiajun Zheng, Fangxin Liu, Jie Zhang, Chen Tian, Chengying Huan

llms agents efficiency

arXiv Machine Learning

Jul 24

Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

arXiv:2607. 21535v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel.

By Alagappan Valliappan

efficiency

Hugging Face Trending Papers

Jul 23

Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel. Frontier models increasingly ship a built-in Multi-Token-Prediction (MTP/NEXTN) draft head under the assumption that the draft is negligibly cheap.

efficiency

arXiv Computation and Language

Aug 24

GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring

arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...

By Xuming Ye, Zeming Ma, Runjie Yu, Yuan Liu, Tianle Li, Shuhan Bai, Jian Zhou, Fei Wu

llms diffusion efficiency

arXiv Machine Learning

3d ago

Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...

By Ephrem Wu

diffusion fine-tuning benchmarks

arXiv AI

3d ago

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

arXiv:2608.30386v1 Announce Type: cross Abstract: Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substa...

By Yanqi Yu, Pingwei Sun, Jianchao Tan, Tao Zhang, Yuchen Xie, Xunliang Cai, Yao Liu

efficiency benchmarks

One email a morning, machine-written

Related stories

Bole: Efficient Tree Speculation for Hybrid-Attention Language Models

Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring

Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving