arXiv Computation and Language
Dual-Slot: From Serial Loops to Parallel Slots in Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Oct 12
Contextual Correction of Parallel Draftsfor Speculative Decoding
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
arXiv AI
Oct 2
Closing the Loop: Practical Training Recipes for Looped Language Models
The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.
arXiv Computation and Language
6d ago
LoopFold: Parallelizing Recurrent Depth in Looped Transformers
arXiv:2610.35556v1 Announce Type: new Abstract: Looped Transformers increase computation per token by repeatedly applying a shared backbone, offering a way to improve language modeling and downstream...
arXiv AI
Aug 14
Dual-Flow Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
arXiv:2608. 12385v1 Announce Type: new Abstract: As large language models serve more requests, cumulative inference cost is becoming increasingly important relative to one-time training cost.
arXiv Computation and Language
Oct 12
One Model to Loop Them All: Elastic Latent Recurrence for Language Models
The paper introduces Elastic Latent Recurrence (ELR), a training recipe that enables looped language models to match the accuracy of non-recurrent models at a single loop while allowing additional loops to improve accuracy with more compute. ELR achieves this through gated state updates with learned step sizes, mixed loop counts during training, and conditioning on loop index and budget. The method also reduces training and inference overhead in Mixture-of-Experts models and produces a single checkpoint, Koi‑4B, that performs competitively with state‑of‑the‑art models on multiple benchmarks.
arXiv AI
Oct 21
Large Flow Language Models
arXiv:2610.28559v1 Announce Type: cross Abstract: Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet ef...