The paper introduces Split Policy Learned Image Tokenization (SPLIT), a method that learns to allocate token resolution based on the model’s interpretation of task-relevant information rather than raw pixel statistics. SPLIT produces shorter token sequences while matching the accuracy of full-resolution Vision Transformers on classification tasks, and it also achieves competitive performance on dense prediction benchmarks such as COCO and ADE20K.
arXiv:2605.12491v2 Announce Type: replace Abstract: Vision Transformers (ViTs) learn rich visual-semantic representations through all-to-all self-attention among patch tokens. However, this design im...
The paper introduces Prune-and-Merge ViT, a training‑free token compression framework for Vision Transformers that combines class‑attention pruning with similarity‑based merging to reduce computational cost while preserving semantic information. It evaluates robustness using two new metrics—Relative Accuracy Drop (RAD) and Prediction Flip Rate (PFR)—and shows that the method maintains or improves accuracy on pretrained ViT and DeiT models compared to existing token reduction techniques. The authors also argue that the hybrid approach does not induce hallucination‑like behavior and keeps predictions stable under common image transformations.
arXiv:2607.09086v2 Announce Type: replace Abstract: We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transfo...
arXiv:2607. 02612v1 Announce Type: cross Abstract: Vision Transformers achieve strong image classification accuracy but process all image regions with nearly the same computation, even when many regions are redundant or uninformative.
DART is a fully differentiable Dynamic Adaptive Region Tokenizer that replaces fixed‑grid tokenizers in vision models. It uses learnable region scores and quantile‑based partitioning to create content‑aware patches of varying sizes, allocating more tokens to information‑rich areas. In experiments, a DART‑equipped DeiT‑Small matches the performance of a larger DeiT‑Base while nearly doubling inference speed, and the approach also benefits dense prediction and video tasks.
arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.
arXiv:2606. 08156v1 Announce Type: cross Abstract: Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.
The paper studies Graph-Guided Token Merging (G2TM), a module that reduces token count in Vision Transformers. It evaluates G2TM across multiple segmentation frameworks and decoder types, finding that its performance gains are tied to the encoder rather than the decoder. The authors report consistent reductions in GFLOPs (22‑47%) and throughput improvements (up to 74%) on ADE20K, with optimal hyperparameters depending mainly on backbone pre‑training and target dataset.
ProgResViT is an input‑adaptive Vision Transformer that processes images progressively across multiple rounds, starting with a low‑resolution image and a narrow subnetwork and refining the prediction with higher resolution and a wider subnetwork if needed. The method introduces Progress‑Conditioned Soft Gating (PSG) to share a single backbone across rounds while conditioning token fusion and layer outputs on the current round, block, and input resolution. Experiments on DeiT show improved accuracy‑compute trade‑offs compared to adaptive‑width, adaptive‑depth, and dynamic‑token baselines, and the design also benefits self‑supervised DINO representations and downstream semantic segmentation.
arXiv:2605. 13178v2 Announce Type: replace-cross Abstract: In large vision-language models, visual tokens typically constitute the majority of input tokens, leading to substantial computational overhead.
arXiv:2605.18177v2 Announce Type: replace Abstract: Query-based Vision Transformer segmentation models typically reconstruct dense spatial feature maps to predict masks, inheriting design patterns fr...
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...