arXiv Computer Vision
Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Aug 31
Token-Budget Distillation: Transferring Full-Token Semantics to Compressed Video Vision-Language Models
Token-Budget Distillation (TBD) is a parameter‑efficient fine‑tuning framework that adapts video vision‑language models to a fixed token budget. It freezes the pretrained backbone, updates only LoRA adapters, and incorporates FlashVID visual token compression. TBD uses a dual‑path teacher‑student design with full‑token supervision and compressed student optimization, enabling the student to recover full‑token semantics while remaining efficient under aggressive token reduction.
arXiv Computer Vision
Aug 26
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
arXiv:2606.18974v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly a...
arXiv Machine Learning
Aug 20
Accelerating Visual On-Policy Distillation with Batched Speculative Jacobi Rollouts
The paper introduces HB‑SJD, a batched Speculative Jacobi Decoding backend that accelerates visual on‑policy distillation (OPD) by allowing images to advance independently and processing multiple tokens in parallel without a draft model. HB‑SJD switches between Full and Compact execution as images finish, reducing rollout and overall training time while maintaining generation quality. Experiments with LlamaGen demonstrate significant speedups without altering the teacher, distillation objective, or optimization procedure.
arXiv Computer Vision
Sep 23
From Token Importance to Conditional Removability: Rethinking Visual Token Pruning in Multimodal Large Language Models
The paper argues that token importance alone is insufficient to determine safe removal of visual tokens in multimodal large language models, because removability depends on representation depth and the surrounding deletion set. Through controlled experiments, the authors show that the same tokens can have different effects when removed at different depths or contexts. They introduce CoRePrune, a training‑free two‑stage pruning framework that refreshes deletion effects as visual representations evolve and refines candidate tokens based on the current deletion set, achieving high performance retention across multiple backbones and reducing prefill time significantly.
Hugging Face Trending Papers
Jun 17
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion.
arXiv Machine Learning
Aug 20
Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies
The paper introduces Role-Conditioned Sub-Token Routing (RoleSub), a method that compresses the value representations of retained tokens in Vision‑Language‑Action models. By partitioning each token’s value into orthogonal groups and routing them based on token features, a latent role, and language context, RoleSub can also compress language values. Experiments on OpenVLA‑OFT‑7B show that, at matched visual‑KV budgets, RoleSub outperforms token‑only control in most settings and reduces total KV to 9.2–11.3% of the original while maintaining strong control performance.