arXiv Computer Vision
Let ViT Speak: Generative Language-Image Pre-training
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Jul 31
Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution
arXiv:2607. 26596v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual understanding within a unified transformer architecture.
arXiv Computation and Language
Aug 28
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
arXiv AI
Jun 3
Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models
arXiv:2605. 18160v2 Announce Type: replace-cross Abstract: In recent years, multimodal large language models (MLLMs) have achieved remarkable progress, primarily attributed to effective paradigms for integrating visual and textual information.
arXiv AI
Jul 29
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
arXiv:2607. 25527v1 Announce Type: cross Abstract: Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities.
arXiv AI
Aug 25
HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
arXiv:2509.23928v3 Announce Type: replace-cross Abstract: Speculative decoding has proven effective for accelerating inference in Large Language Models (LLMs), yet its extension to Vision-Language Mo...
arXiv AI
Jul 7
Full Glyph Images Beat Token Embeddings: A Controlled Study for Transformers
arXiv:2607. 03994v1 Announce Type: cross Abstract: Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned.