arXiv Computer Vision
Consomm\'e: Conditioned Token Merging for Split Inference
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Jul 7
Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers
arXiv:2607. 02612v1 Announce Type: cross Abstract: Vision Transformers achieve strong image classification accuracy but process all image regions with nearly the same computation, even when many regions are redundant or uninformative.
arXiv Machine Learning
Aug 14
MergeOver: Post-Training Token Merging for Recursive Vision Transformers
arXiv:2608. 13141v1 Announce Type: cross Abstract: Vision Transformers (ViTs) demonstrate exceptional performance in computer vision but suffer from large parameter counts and quadratic computational complexity, severely limiting their deployment on resource-constrained edge hardware.
arXiv AI
Oct 7
Later Is Better: Token Reduction for ViTs Under Distribution Shift
arXiv:2610.07758v1 Announce Type: cross Abstract: Training-free token reduction accelerates vision transformers by removing redundant tokens across layers, recovering most of the original accuracy at...
arXiv Computer Vision
Oct 27
Rescuing Your Pruning: Rethinking Visual Token Pruning and Post-Training Recovery in MLLMs
arXiv:2610.33688v1 Announce Type: new Abstract: Visual token pruning substantially reduces the inference cost of multimodal large language models (MLLMs), yet existing methods are mostly training-fre...
arXiv Computer Vision
Sep 17
Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM
The paper studies Graph-Guided Token Merging (G2TM), a module that reduces token count in Vision Transformers. It evaluates G2TM across multiple segmentation frameworks and decoder types, finding that its performance gains are tied to the encoder rather than the decoder. The authors report consistent reductions in GFLOPs (22‑47%) and throughput improvements (up to 74%) on ADE20K, with optimal hyperparameters depending mainly on backbone pre‑training and target dataset.
arXiv Computer Vision
Oct 12
LapMerge: Differentiable Exact-Budget Token Merging in Linear Time
LapMerge is a new differentiable token merging operator that partitions an ordered sequence into exactly N equal‑mass segments using a learned Laplace‑kernel density, then aggregates each segment with prefix/suffix scans, achieving an exact output budget in linear time. The method preserves information while allowing end‑to‑end differentiability and can change the budget without retraining. Experiments on vision and vision‑language tasks show LapMerge matches or outperforms existing reducers and can be up to two orders of magnitude faster for long sequences, especially when token order reflects spatial or temporal locality.