Efficient multimodal inference is increasingly constrained not only by model quality or FLOP count, but also by the cost of preserving, moving, routing, caching, and quantizing multimodal representations under latency, memory, and energy constraints. This paper reviews recent advances in efficient vision-language and multimodal large language models, covering visual token compression, video token management, KV-cache optimization, Mixture-of-Experts (MoE) routing, low-bit quantization, edge deployment, and hardware-aware benchmarking.
arXiv:2607. 20981v1 Announce Type: new Abstract: Efficient multimodal inference is increasingly constrained not only by model quality or FLOP count, but also by the cost of preserving, moving, routing, caching, and quantizing multimodal representations under latency, memory, and energy constraints.
By Jay Gor, Karm Dave, Akshita Abrol, Rajesh Gupta, Sudeep Tanwar, Zhengkui Wang
The paper introduces EMMI, a framework that enables communication‑efficient inference of multimodal large language models (MLLMs) on edge devices. EMMI encodes each sensor modality separately, fuses the representations, and compresses them into a compact latent vector that is transmitted to a server for high‑capacity reasoning. Experiments on a multimodal benchmark show that EMMI can cut the communication payload by 32× while keeping accuracy comparable, achieving up to a 3.4× reduction in end‑to‑end inference latency under bandwidth‑constrained conditions.
By Motahare Mounesan, Irfan Khan
arXiv:2508.03351v3 Announce Type: replace-cross
Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse language tasks, motivating their extension to vision-la...
By Yufei Xue, Yushi Huang, Lunjie Zhu, Jiawei Shao, Jun Zhang
arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.
By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin
arXiv:2609.26425v1 Announce Type: new
Abstract: KV cache memory has become a major deployment bottleneck for video generation and world models, which motivates low-bit quantization study for efficien...
By Jiaqi Zhao, Xiaobin Hu, Bo Yin, Junpeng Jiang, Miao Zhang, Shuicheng Yan
The paper introduces Llama-Mobile, a framework that quantizes vision‑language models for efficient mobile deployment. It uses a quantization pipeline that generates training data from the model itself, eliminating the need for the original training setup, and employs a novel 2.7‑bit‑per‑parameter format optimized for Arm CPUs. Applying this method, the authors compress the Llama 3.2 11B Vision Instruct model to 3.7 GB with 8‑bit activations while maintaining strong performance on visual question answering tasks.
By Luka Ribar, Jeevan Bhoot, Douglas Orr
The paper introduces a token‑oriented semantic communication framework that transmits only task‑relevant image latents instead of full token embeddings, reducing communication cost and improving interoperability. It leverages a spatial alignment between vision transformer patch tokens and learned image compression latents, enabling token‑level relevance estimation and selective transmission. Experiments on ImageNet demonstrate a superior rate–accuracy trade‑off compared to existing semantic communication methods and hand‑crafted codecs.
By Jiwoong Im, Minwoo Kim, Jaeho Lee, Yo-Seb Jeon, Yongjune Kim
arXiv:2609.16689v1 Announce Type: new
Abstract: Large-scale vision-language models (VLM) such as CLIP enable strong open-vocabulary reasoning, yet deploying these capabilities on resource-constrained...
By Jinwoo Jeon, GyuYeop Do, Yubin Lim, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, Byung-Jun Lee
Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge devices encode visual inputs into tokens for a general-purpose cloud MLLM. However, dense visual-token sequences increase cloud-side inference costs.
Tree-VQ introduces a progressive tree‑structured vector quantization framework for learned image compression, organizing discrete codewords in a hierarchical binary tree where each latent token is represented by a routed root‑to‑leaf path. Every prefix of this path yields a valid quantized representation, enabling coarse reconstructions from shallow nodes and successive refinements from deeper nodes. The method incorporates a prefix‑compatible tree entropy model, rate‑aware refinement scheduling, and hierarchical prefix supervision to achieve efficient, low‑latency compression with superior perceptual quality and fewer parameters compared to existing approaches.
By Xinkun Wang, Tianyi Xu, Qingyu Luo, Mingming Ma, Changzhe Jiao, Fu Li, Yi Niu
The paper introduces Channel Group-Shared (CGS) low‑rank approximation, a Singular Value Decomposition–based strategy that shares down/up‑projection matrices across channel groups while using lightweight diagonal matrices for each group. This design dramatically cuts the parameter count of pointwise convolutions, which dominate the size of large‑kernel CNNs such as RepLKNet, ConvNeXt, and SLaK. Experiments show that CGS‑enhanced models maintain competitive accuracy while substantially reducing storage, memory bandwidth, and loading latency, making them viable for deployment on edge devices.
By Hao Luo, Yiting Yang, Wenyi Zhao, Man Jiang, Zhijun Lin, Ghulam Mohiuddin, Ting Jiang, Kunming Luo, Zihao Zhang, Qingsen Yan, Guoqing Wang, Wei Dong, Peng Wang