LumoTree is a verifier designed for hybrid language models that ensures a single coherent continuation across recurrent, convolution, and attention states. It executes recurrent paths in parallel, reuses state tiles, and coordinates replay, history gathering, and cache remapping through a shared logical tree. Experiments demonstrate exact candidate‑selection parity and recurrent agreement within error bounds, and a Qwen3.8‑27B deployment achieved 25.63 pooled tokens/s on a single NVIDIA DGX Spark.
arXiv:2608. 01651v1 Announce Type: cross Abstract: Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memory-bound.
arXiv:2609.23900v1 Announce Type: new Abstract: Tree speculative decoding verifies multiple candidate continuations in one target forward pass. For attention-only transformers, the verifier mainly ne...
arXiv:2609.37532v1 Announce Type: cross Abstract: Growing large language model applications demand efficient inference. At high concurrency, block-diffusion speculative decoding suffers from verifica...
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.
arXiv:2606. 01774v1 Announce Type: cross Abstract: Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-latency deployment.
arXiv:2609.24698v1 Announce Type: new Abstract: Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which foll...
DRelay introduces a global draft context mechanism to improve prefix-aware parallel speculative decoding for large language models. By using a global reader to extract predictive information across the entire draft block and a causal selector to repair early token selection errors, DRelay extends the accepted prefix length and enhances decoding performance. Experiments on eight benchmarks show consistent gains over existing methods such as DFlash, Domino, and DSpark, with notable speedup improvements in SGLang serving.
arXiv:2608. 04962v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck.
arXiv:2607. 14952v1 Announce Type: new Abstract: A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment.
arXiv:2609.07108v1 Announce Type: new Abstract: Speculative decoding accelerates rollout generation, which dominates the cost of reinforcement learning (RL) post-training. Online co-training can furt...
arXiv:2606. 01019v1 Announce Type: cross Abstract: Large Language Model (LLM) generation remains expensive because autoregressive decoding calls the model once for each new token.
arXiv:2607. 20467v1 Announce Type: new Abstract: While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.