← Back to all news
arXiv Computer Vision By Yisen Wang, Zhirong Wu, Limin Wang

CoViT: Instance-Correspondence Contrastive Learning for Vision Transformer

Read the original on arXiv Computer Vision →

CoViT introduces a self‑supervised framework that enhances Vision Transformers with instance‑aware representations by leveraging geometry‑guided contrastive learning. It refines attention maps to generate instance masks and constructs triplets that mine the hardest intra‑ and inter‑instance examples, driving a contrastive loss that reduces intra‑instance variance while increasing inter‑instance margins. The method yields consistent AP gains of over 2 points on instance‑level tasks without requiring extra decoders or labels.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Oct 16

V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance

The paper introduces V‑GIFT, a lightweight method that enhances visual instruction tuning for multimodal large language models by incorporating a small set of visually grounded self‑supervised tasks expressed as natural language instructions. These tasks—such as rotation prediction, color matching, and cross‑view correspondence—are reformulated into image‑instruction‑response triplets, providing supervision that cannot be solved without visual evidence. The approach requires no human annotations, architectural changes, or extra training stages, and consistently improves vision‑centric performance across various models, regimes, and benchmarks when only 3‑10% of the training data is augmented with these instructions.

By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris
llmsfine-tuningmultimodalbenchmarks
More like this →