arXiv Computation and Language
Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 16
Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
arXiv:2505. 18227v4 Announce Type: replace-cross Abstract: In Transformer architectures, tokens\textemdash discrete units derived from raw data\textemdash are formed by segmenting inputs into fixed-length chunks.
arXiv AI
Jul 29
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
arXiv:2607. 25527v1 Announce Type: cross Abstract: Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities.
arXiv Computer Vision
3d ago
Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces
arXiv:2609.40362v1 Announce Type: new Abstract: We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models either model both language and q...
arXiv Computation and Language
Sep 21
VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits
arXiv:2505.10202v2 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved remarkable success but face significant computational and memory challenges, particularly due to their e...
arXiv AI
Sep 10
MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
MoEMB introduces a mixture‑of‑experts (MoE) approach to scale universal multimodal embeddings (UME) without increasing the size of the output vector or relying on autoregressive decoding. By expanding encoder capacity along the expert axis, MoEMB achieves state‑of‑the‑art performance on MMEB‑V2 and MRMR benchmarks with only 3 B active parameters, outperforming TTE‑based methods that use more than four times as many active parameters and require significantly more compute. The paper also presents the first comprehensive study of adaptive computation for MoE‑based embeddings, exploring training‑time and inference‑time strategies to further improve efficiency for large‑scale retrieval and recommendation systems.
arXiv AI
Sep 18
M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models
The paper introduces M2Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss for continuous action signals by decomposing latent features into multiple heads and assigning independent codebooks to each. This design expands representational expressivity, leading to lower reconstruction error and higher success rates in Vision‑Language‑Action models evaluated on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks. Ablation studies confirm the effectiveness of both multi‑head and multi‑codebook mechanisms.