Vision Transformer architectures have showcased impressive performance on semantic segmentation, yet their quadratic attention complexity hinders practical applications. Recent works have focused on m...
The paper studies Graph-Guided Token Merging (G2TM), a module that reduces token count in Vision Transformers. It evaluates G2TM across multiple segmentation frameworks and decoder types, finding that its performance gains are tied to the encoder rather than the decoder. The authors report consistent reductions in GFLOPs (22‑47%) and throughput improvements (up to 74%) on ADE20K, with optimal hyperparameters depending mainly on backbone pre‑training and target dataset.
The paper introduces Prune-and-Merge ViT, a training‑free token compression framework for Vision Transformers that combines class‑attention pruning with similarity‑based merging to reduce computational cost while preserving semantic information. It evaluates robustness using two new metrics—Relative Accuracy Drop (RAD) and Prediction Flip Rate (PFR)—and shows that the method maintains or improves accuracy on pretrained ViT and DeiT models compared to existing token reduction techniques. The authors also argue that the hybrid approach does not induce hallucination‑like behavior and keeps predictions stable under common image transformations.
arXiv:2606. 08156v1 Announce Type: cross Abstract: Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.
arXiv:2610.21207v1 Announce Type: new Abstract: Point-cloud streams are central to autonomous driving and robotics, yet existing segmentation networks process every frame from scratch, although conse...
arXiv:2605.18177v2 Announce Type: replace Abstract: Query-based Vision Transformer segmentation models typically reconstruct dense spatial feature maps to predict masks, inheriting design patterns fr...
3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details.
The paper introduces Feature Interaction Network (FINE), a lightweight semantic alignment module for feature fusion networks in object detectors. FINE refines low‑level features using high‑level contextual guidance through cross‑level attention, and employs Alignment‑Aware Token Sampling to reduce attention complexity. The resulting spatial‑channel modulation map selectively enhances semantically relevant pixels while preserving sub‑pixel localization, leading to improved detection accuracy with minimal computational overhead.
Pixels to Tokens introduces TEAL, a token‑space efficient active learning framework that operates at the pixel level for semantic segmentation using a frozen Vision Transformer backbone. TEAL selects diverse candidate pixels via a two‑level MaxHerding strategy on multi‑layer token descriptors and refines uncertainty on a finer decoder grid, avoiding interpolation artifacts. Across multiple datasets, TEAL outperforms prior baselines under extreme label scarcity, achieving up to +24.36 mIoU with a single labeled pixel per image and reaching 97% of fully supervised performance with only ten pixels.
VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.
arXiv:2605. 13178v2 Announce Type: replace-cross Abstract: In large vision-language models, visual tokens typically constitute the majority of input tokens, leading to substantial computational overhead.
arXiv:2607. 02612v1 Announce Type: cross Abstract: Vision Transformers achieve strong image classification accuracy but process all image regions with nearly the same computation, even when many regions are redundant or uninformative.