Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
Read the original on arXiv Machine Learning →The paper introduces Learned Relay Representations (Relay), a technique for Masked Diffusion Models (MDMs) that preserves and forwards internal latent information across denoising steps via a differentiable per-token channel trained with truncated backpropagation through time. Relay enables MDMs to be forward‑thinking, avoiding costly recomputation of internal representations and improving efficiency. Applied to Fast‑dLLM v2, Relay outperforms standard supervised fine‑tuning on coding tasks and reduces inference latency by up to 32%, demonstrating a clear performance‑latency advantage for diffusion language models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.