arXiv Computation and Language August 28, 2026 By Insu Lee, Wooje Park, Wonseok Shin, Jinwoo Son, Byonghyo Shim

Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models

Read the original on arXiv Computation and Language →

Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 11

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

arXiv:2605. 25820v2 Announce Type: replace Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel.

By Yulin Yuan, Hongshuo Zhao, Xiangming Meng

llms diffusion multimodal benchmarks

arXiv Computation and Language

1d ago

TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs

arXiv:2607.28640v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) should exhibit cross-modal consistency, generating consistent responses given semantically equivalent inpu...

By Andong Hua, Colton Bishop, Igor Mordatch, Arian Hosseini, Jindong Gu, Aleksandra Faust, Rebecca Roelofs, Yao Qin

llms multimodal benchmarks safety

arXiv AI

Jul 10

Reinforcing the Generation Order of Multimodal Masked Diffusion Models

arXiv:2607. 08056v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) have recently achieved substantial progress in natural language generation tasks.

By Yidong Ouyang, Zhe Wang, Sourav Bhabesh, Dmitriy Bespalov

llms diffusion multimodal benchmarks safety

arXiv AI

Aug 25

HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models

arXiv:2509.23928v3 Announce Type: replace-cross Abstract: Speculative decoding has proven effective for accelerating inference in Large Language Models (LLMs), yet its extension to Vision-Language Mo...

By Zhinan Xie, Peisong Wang, Shuang Qiu, Jian Cheng

llms multimodal benchmarks safety

Hugging Face Trending Papers

Jul 9

Reinforcing the Generation Order of Multimodal Masked Diffusion Models

Diffusion Language Models (DLMs) have recently achieved substantial progress in natural language generation tasks. Recent research demonstrates that adaptive token generation ordering can significantly improve performance in mathematical reasoning and code synthesis applications.

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 28

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

arXiv:2607. 22586v1 Announce Type: new Abstract: Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a major bottleneck due to the large number of visual tokens.

By Jinsong Shu, Chenyang Wu, Zhongle Xie, Baokun Wang, Lidan Shou

llms efficiency multimodal benchmarks

One email a morning, machine-written

Related stories

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs

Reinforcing the Generation Order of Multimodal Masked Diffusion Models

HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models

Reinforcing the Generation Order of Multimodal Masked Diffusion Models

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models