Hugging Face Trending Papers
CODI: Improving Self-Supervised Vision Transformers with Cross Distillation
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computer Vision
2d ago
CODI: Improving Self-Supervised Vision Transformers with Cross Distillation
The paper introduces CODI, a method that enhances self‑supervised Vision Transformers by mixing a small portion of the teacher’s class‑token distribution into each teacher patch‑token target. This cross distillation approach gives every masked patch an attention‑independent semantic correction without altering the architecture. Experiments demonstrate that CODI improves performance on both dense prediction and image‑level tasks under aligned pretraining settings.
arXiv Computer Vision
Sep 1
Enhancing Self-Supervised Visual Representation Learning via Low-Rank Adapted LLMs
arXiv:2507.00754v3 Announce Type: replace Abstract: The integration of Large Language Model (LLMs) blocks with Vision Transformers (ViTs) holds immense promise for vision-only tasks by leveraging the...
arXiv Computer Vision
Oct 2
Subtoken Vision Transformer for Fine-grained Recognition
arXiv:2607.09086v2 Announce Type: replace Abstract: We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transfo...
Hugging Face Trending Papers
Aug 11
Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity
Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.
arXiv AI
Jun 2
On the Limits of Token Reduction for Efficient Unified Vision Language Training
arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.
arXiv AI
Sep 30
Representation by Design in Generation: Cross-View Class-Token Alignment in Diffusion Transformers
arXiv:2609.36348v1 Announce Type: cross Abstract: Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas th...