arXiv AI
Aug 11
Full-bandwidth transformer
arXiv:2608. 08888v1 Announce Type: new Abstract: Autoregressive transformers compute along two axes: horizontally across generated tokens, and vertically through model depth.
SNLP: Layer-Parallel Inference via Structured Newton Corrections
arXiv:2605. 17842v3 Announce Type: replace Abstract: Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipeline parallelism.
Related stories
arXiv AI
Jul 14
Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers
arXiv:2602. 17993v2 Announce Type: replace-cross Abstract: Complex problems, whether in math, logic, or planning, are solved by humans through a sequence of steps where the result of one step informs the next.
Hugging Face Trending Papers
Jul 9
Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models
Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbone fixed.
arXiv Machine Learning
Sep 25
FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates
FlashLoop is a training‑free inference framework for Looped Transformers that reduces cross‑loop redundancy by employing token‑sparse updates, sparse attention, and KV‑residual quantization. It exploits observations that, as loops progress, state changes concentrate on a small token subset, attention differences are dominated by a sparse key subset, and KV residuals become amenable to low‑bit quantization. The method achieves lossless accuracy with up to 1.64× speedup and 6× KV‑cache memory reduction across several Looped Transformer models.
arXiv Machine Learning
Sep 2
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
arXiv:2605.26797v2 Announce Type: replace Abstract: We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden...
arXiv Machine Learning
Jun 10
PRISM: Parallel Residual Iterative Sequence Model
arXiv:2602. 10796v3 Announce Type: replace Abstract: Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models.
arXiv AI
Jun 3
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem
arXiv:2602. 20217v2 Announce Type: replace-cross Abstract: Self-speculative decoding (SSD) accelerates LLM inference by skipping layers to create an efficient draft model, yet existing methods often rely on static heuristics that ignore the dynamic computational overhead of attention in long-context scenarios.
arXiv AI
Jul 24
Scaling Interpretable Transformers with Parity Bottleneck Layers
arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.
arXiv AI
Sep 15
Carryover Drafting: Recycling Rejected States for Speculative Decoding
arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...
arXiv AI
Jun 9
WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing
arXiv:2606. 07710v1 Announce Type: cross Abstract: The autoregressive nature of large language models (LLMs) remains a significant bottleneck for inference, particularly in complex agentic workloads.
arXiv Machine Learning
Jun 18
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.
arXiv AI
Sep 3
CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.