← Back to all news
arXiv Machine Learning By Yasir Mehmood, Kashif Javed

Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders

Read the original on arXiv Machine Learning →

The paper introduces ZENDAYA, a closed‑form bandwidth dial that replaces fixed wait‑k rules in streaming multimodal decoders with a single continuous parameter γ. This schedule ties the visible source prefix to generation progress and the input’s predicted length, allowing a single offline decoder to serve as both offline and real‑time streaming models. Experiments on video‑to‑text and audio‑to‑text tasks show that reading less of the source can improve text quality, with significant METEOR gains across three public corpora.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 17

Look Less, Hear Better: Jointly Rewarded GRPO for Streaming ASR

The paper introduces AWED, a word‑level emission‑delay metric, and demonstrates that post‑training a delayed‑streaming model with a joint reward (GRPO) improves both accuracy and latency. Using a single operating point (τ=6 frames), the method outperforms both its supervised baseline and the Voxtral Realtime backbone across all lookahead budgets, reducing WER by up to 30.8% at 80 ms delay and lowering median AWED from 1.17 s to 1.04 s.

By Xiuwen Zheng
llmsfine-tuningmultimodalsafety
More like this →
arXiv Machine Learning
3d ago

The Lattice of Transition Laws

The paper introduces a framework that unifies diffusion and autoregressive (AR) generative models as paths on a corruption lattice, defining a cost for decoding schedules based on discarded parallel steps. It shows that the minimal number of steps for a zero‑cost schedule is determined by the data’s geometry—specifically, the graph’s treedepth for Markov data—providing a theoretical limit for both token and continuous field models. By estimating pairwise dependencies from pretrained weights, the authors predict the ranking of decoding schedules before execution, and validate these predictions across text, image, and video generation tasks.

By T. Y. Tsui, Jiatao Gu, Lingjie Liu
diffusionbenchmarks
More like this →
arXiv AI
Sep 2

Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models

The paper introduces GLANCE, a one‑pass block drafting method that enables lossless speculative decoding for vision‑language models. By using a block‑diffusion head that reads the fused vision‑language state, GLANCE eliminates the need for the drafter to process the image at every step, allowing it to fill an entire block in a single forward pass. Experiments show that GLANCE can decode up to 2.93× faster than autoregressive decoding while maintaining exact greedy decoding results across multiple tasks.

By Jungseob Lee, Seongtae Hong, Dongyub Jude Lee, Chanjun Park, Jaehyung Seo, Sugyeong Eo, Heuiseok Lim
llmsdiffusionmultimodal
More like this →