arXiv AI
AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 11
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
arXiv:2606. 12412v1 Announce Type: cross Abstract: Vision-language models (VLMs) project images into hundreds to thousands of visual tokens, making decoder inference expensive in both attention computation and KV-cache memory.
arXiv AI
Jul 1
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
arXiv:2606. 31903v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation.
arXiv AI
Jun 9
Late-Layer Fusion is Enough: Dual-Path Vision Token Routing for Multimodal Large Language Models under Visual Saturation
arXiv:2606. 09131v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) commonly inherit the deep, symmetric Transformer backbone designed for unimodal text modeling, and apply the same computation uniformly to image and language tokens.
arXiv Machine Learning
Aug 20
Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies
The paper introduces Role-Conditioned Sub-Token Routing (RoleSub), a method that compresses the value representations of retained tokens in Vision‑Language‑Action models. By partitioning each token’s value into orthogonal groups and routing them based on token features, a latent role, and language context, RoleSub can also compress language values. Experiments on OpenVLA‑OFT‑7B show that, at matched visual‑KV budgets, RoleSub outperforms token‑only control in most settings and reduces total KV to 9.2–11.3% of the original while maintaining strong control performance.
Hugging Face Trending Papers
Aug 19
Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies
The paper introduces Role-Conditioned Sub-Token Routing (RoleSub), a method that compresses the value representations of retained tokens in Vision‑Language‑Action models. By partitioning each token’s value into orthogonal groups and routing them based on token features, latent roles, and language context, RoleSub can compress both visual and language representations without discarding tokens. Experiments on OpenVLA‑OFT‑7B show that, at matched visual‑KV budgets, RoleSub outperforms token‑only control in most settings and can reduce total KV to 9.2–11.3% of the original while maintaining strong control performance.
arXiv Machine Learning
Aug 10
An AI4AI Framework for Visual Token Pruning
arXiv:2608. 07193v1 Announce Type: new Abstract: Visual-token pruning can substantially reduce the inference cost of multimodal large language models (MLLMs), yet existing methods largely rely on fixed, handcrafted heuristics and costly expert trial and error.