arXiv AI By Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar, Jaydeep Sen, Sachindra Joshi

Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI
Jun 9

MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework

arXiv:2606. 07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language Models (VLMs), but this design makes deployment expensive in both storage and computational overhead.

By Haowen Xiang, Yibo Yan, Jiahao Huo, Yu Huang, Yi Cao, Mingdong Ou, Xuming Hu
arXiv Computation and Language
6d ago

OpticalRAG: Pixel-Space Compression for Token-Efficient Retrieval-Augmented Generation

OpticalRAG is a retrieval‑augmented generation framework that compresses documents by rendering them as images and encoding them with a frozen vision encoder. It retrieves visual tokens using an Encoder‑Bridged Retrieval mechanism and condenses them into as few as 16 query‑conditioned prefix tokens via Query‑Driven Distillation, avoiding costly cross‑modal training. The method achieves state‑of‑the‑art compression‑aware accuracy on LongBench‑v2, BAMBOO, and LooGLE‑v2 while adding only 16 tokens to the LLM input.

By Senhao Liu, Yuheng Zhang, Chunyu Wei, Yueguo Chen, Xinran Zhang
arXiv AI
Oct 7

VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs

VisionWeave introduces elastic visual representation weaving, a native capability for multimodal large language models that learns where and at what granularity to encode visual information. The method combines a gated spatial pooler for coarse representations with a granularity router that allocates content‑adaptive token usage, trained end‑to‑end on large‑scale data. Experiments on Qwen3.5‑4B and Qwen3.8‑27B show that VisionWeave can save 43.0% of tokens while preserving 98.9% of performance across eight benchmarks, and delivers significant throughput gains and latency reductions when deployed on the SGLang serving engine.

By Yuan Feng, Qize Yang, Ruizhe Chen, Sibo Song, Haolin He, Muzhi Zhu, Zihan Liu, Yunfei Chu, Xize Cheng, Yuxuan Wang, Jin Xu, Xike Xie