AutoIndexer is a generative model that trains causal models on chains of edits, enabling flexible post‑generation modifications such as token insertions, substitutions, and deletions. It introduces marker tokens for opening and closing edits and moves a cursor through the token sequence, mimicking a text editor. By perturbing ground‑truth labels with inverse edit operations during training, AutoIndexer becomes robust to erroneous context and can perform editing tasks like code correction while maintaining performance on standard LLM benchmarks.
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
arXiv:2609.20830v1 Announce Type: new Abstract: Revision-capable generation is appealing because it can insert or revise earlier content, but many non-autoregressive and edit-based approaches obtain...
arXiv:2609.20830v2 Announce Type: replace Abstract: Language models write text once, left to right, and models that write in other orders, such as diffusion LMs, pay for it with many full-sequence pa...
arXiv:2607. 22663v1 Announce Type: new Abstract: Block diffusion has emerged as the dominant paradigm for scaling discrete diffusion language models (dLLMs), because decoding text in fixed-size blocks preserves parallel generation within each block while keeping the quadratic attention cost tractable.
Diff Diffusion is a diffusion-inspired large language model framework designed for efficient code editing. It predicts edit paths from an original snippet to its revised form by representing denoising as structured edit operations, and uses multi-stage training and efficient parallel decoding to focus computation on changed regions. The approach matches or outperforms autoregressive baselines while achieving up to 2.78× faster inference and a 39.0% reduction in median model latency in production.
Causal Edit Serialization (CES) is a framework that enables autoregressive language models to perform revisions—such as insertion, deletion, replacement, copying, and movement—within the same causal token stream. CES uses special pointer tokens paired with predicted source addresses, a lightweight localization head, and dual-index rotary position encoding to localize edits while preserving standard causal attention. Experiments show that CES models can execute complete edits with only a modest increase in clean-text bits per byte, and that post‑training with supervised or reinforcement learning can produce beneficial grammatical error corrections.
arXiv:2607. 26504v1 Announce Type: new Abstract: Many discrete reasoning tasks, such as code generation, are inherently non-causal: programmers move between high-level structure and local details, a process we call any-order inference.
arXiv:2605. 26436v2 Announce Type: replace-cross Abstract: Discrete masked diffusion language models such as LLaDA generate text through iterative denoising, where mask tokens are progressively replaced with predicted tokens.
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
InComeS is a framework that improves large language models (LLMs) for model editing by compressing editing contexts into a key‑value cache associated with a special gist token. It adds cross‑attention modules to dynamically select the most relevant information from these gist pools, allowing efficient handling of multiple edits beyond the LLM’s context window. Experiments on diverse editing benchmarks show that InComeS enhances both effectiveness and efficiency compared to existing methods.
arXiv:2606. 16847v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality.
Many discrete reasoning tasks, such as code generation, are inherently non-causal: programmers move between high-level structure and local details, a process we call any-order inference. For autoregressive language models, which lack a native any-order interface, non-causal abilities such as infilling and next-edit prediction require hand-designed mechanisms.