arXiv Computation and Language
Distilling Sequential Computation in Transformer Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
Hugging Face Trending Papers
5d ago
Distilling Sequential Computation in Transformer Language Models
Transformer language models process sequences token by token in an autoregressive manner, making growing contexts increasingly expensive. Yet many adjacent token spans are highly predictable or freque...
arXiv AI
Jun 15
Residual Context Diffusion Language Models
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
arXiv Machine Learning
Sep 2
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
arXiv:2605.26797v2 Announce Type: replace Abstract: We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden...
arXiv AI
Jul 8
DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
arXiv:2607. 06523v1 Announce Type: new Abstract: Long-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation.
arXiv AI
Jul 31
ARC-Encoder: learning compressed text representations for large language models
arXiv:2510. 20535v2 Announce Type: replace-cross Abstract: Recent techniques such as retrieval-augmented generation or chain-of-thought reasoning have led to longer contexts and increased inference costs.
arXiv Machine Learning
Jul 10
Prompt Compression via Activation Aggregation
arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.