arXiv AI
Jun 29
ELF: Embedded Language Flows
arXiv:2605. 10938v2 Announce Type: replace-cross Abstract: Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.
ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
Related stories
arXiv AI
3d ago
Semantic Flow Regularization: Teaching LLMs to Generate Diverse Yet Coherent Responses
arXiv:2605.27971v2 Announce Type: replace-cross Abstract: When large language models are fine-tuned to generate persona- or tone-conditioned responses, their output diversity is severely limited--a f...
arXiv AI
Jun 9
End-to-End Training for Discrete Token LLM based TTS System
arXiv:2606. 09234v1 Announce Type: cross Abstract: Recent state-of-the-art (SOTA) text-to-speech (TTS) systems typically adopt a cascaded pipeline consisting of a speech tokenizer, an autoregressive large language model (LLM), and a diffusion based flow-matching (FM) model, with these components trained independently.
arXiv Machine Learning
3d ago
Learning Representations through Token Prediction: Geometry, Approximation, and Downstream Guarantees
The paper investigates why token prediction, a common pre‑training objective for language models, yields useful representations. It introduces a statistical framework linking token prediction accuracy to the geometry of token embeddings, showing that accurate predictions organize embeddings according to Hellinger distances between context distributions. The authors also propose a self‑consistency principle that refines contextual representations through repeated application of a shared block, and provide downstream guarantees for token generation, community recovery, and linear classification.
arXiv Machine Learning
Aug 28
Squeezing More from Limited Data with Recursive Transformers
The paper investigates how to effectively pre‑train language models when the data budget is limited but compute is plentiful. It shows that increasing model size only improves performance up to an optimal point, after which overfitting degrades generalization, and that this optimal size varies with both the data budget and downstream tasks. To overcome the inefficiencies of standard Transformers in this regime, the authors propose recursive Transformers that reuse a shared block across depth and employ factorized embeddings, achieving better results than standard models on 10M–100M word pre‑training budgets and competitive performance with BabyLM Challenge 2025 winners.
arXiv AI
Jun 16
Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
arXiv:2606. 16847v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality.
arXiv Machine Learning
Jul 14
CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting
arXiv:2602. 03564v2 Announce Type: replace Abstract: Time series forecasting can be viewed as a generative problem that requires both semantic understanding over contextual conditions and stochastic modeling of continuous temporal dynamics.
arXiv Machine Learning
3d ago
Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization
arXiv:2608.30315v1 Announce Type: new Abstract: Token embeddings are the basic representational units that connect discrete tokens with continuous computation in language models. Although modern lang...
arXiv AI
Jun 2
You Can Learn Tokenization End-to-End with Reinforcement Learning
arXiv:2602. 13940v2 Announce Type: replace-cross Abstract: Tokenization is a hardcoded compression step which remains in the training pipeline of Large Language Models (LLMs), despite a general trend towards architectures becoming increasingly end-to-end.
arXiv Machine Learning
Jun 16
Mean-Field Parallel Decoding for Discrete Diffusion Language Models
arXiv:2606. 15805v1 Announce Type: new Abstract: Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding.
arXiv AI
Jun 9
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
arXiv:2606. 08417v1 Announce Type: cross Abstract: Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.
arXiv Machine Learning
Jun 5
A Survey on Diffusion Language Models
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.