arXiv AI
Aug 11
Full-bandwidth transformer
arXiv:2608. 08888v1 Announce Type: new Abstract: Autoregressive transformers compute along two axes: horizontally across generated tokens, and vertically through model depth.
SNLP: Layer-Parallel Inference via Structured Newton Corrections
arXiv:2605. 17842v3 Announce Type: replace Abstract: Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipeline parallelism.
Related stories
arXiv AI
Jul 14
Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers
arXiv:2602. 17993v2 Announce Type: replace-cross Abstract: Complex problems, whether in math, logic, or planning, are solved by humans through a sequence of steps where the result of one step informs the next.
Hugging Face Trending Papers
Jul 9
Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models
Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbone fixed.
arXiv Machine Learning
Jun 10
PRISM: Parallel Residual Iterative Sequence Model
arXiv:2602. 10796v3 Announce Type: replace Abstract: Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models.
arXiv AI
Jun 3
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem
arXiv:2602. 20217v2 Announce Type: replace-cross Abstract: Self-speculative decoding (SSD) accelerates LLM inference by skipping layers to create an efficient draft model, yet existing methods often rely on static heuristics that ignore the dynamic computational overhead of attention in long-context scenarios.
arXiv AI
Jul 24
Scaling Interpretable Transformers with Parity Bottleneck Layers
arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.
arXiv AI
Jun 9
WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing
arXiv:2606. 07710v1 Announce Type: cross Abstract: The autoregressive nature of large language models (LLMs) remains a significant bottleneck for inference, particularly in complex agentic workloads.
arXiv Machine Learning
Jun 18
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.
arXiv Machine Learning
Jun 3
Why Are Linear RNNs More Parallelizable?
arXiv:2603. 03612v3 Announce Type: replace Abstract: The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability.
arXiv Machine Learning
Jul 7
Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers
arXiv:2607. 04819v1 Announce Type: new Abstract: Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks.
arXiv AI
Jun 29
The Context-Ready Transformer
arXiv:2606. 27538v1 Announce Type: cross Abstract: We introduce the context-ready transformer, a new recurrent neural network architecture built from a D-layer transformer block that pre-contextualizes each token before it enters the block.
arXiv AI
1d ago
Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
arXiv:2608. 12385v2 Announce Type: replace Abstract: As large language models serve ever more requests, cumulative inference cost is growing relative to the one-time cost of training.