The paper introduces a method to retrofit depth recurrence into pretrained vision‑language models, reducing the language backbone’s parameter count by pruning layers and looping over the remaining ones with linear adapters. A three‑stage recovery training—language‑only pretraining, vision‑language projector alignment, and full fine‑tuning—helps recover performance, with distillation from the non‑recurrent model further improving results. Experiments on TinyLLaVA‑1.4B show the retrofitted recurrent model outperforms the unpruned baseline while saving 23.3% of language parameters and performs better on reasoning‑oriented benchmarks.
By Madhav Iyengar, Andreas Geiger, Hilde K\"uhne
LoopLook introduces a lightweight encoder-side recurrent perception framework that equips pretrained Vision‑Language Models with adaptive visual depth. By repeatedly revisiting a middle‑to‑late window of the vision encoder and injecting question representations at each loop, it refines visual evidence in a task‑directed manner. The learned stopping mechanism and token‑wise Mixture‑of‑Looks allow each visual token to selectively integrate information across loops, yielding consistent performance gains across multiple backbones and benchmarks.
By Wencheng Ye, Anning Hu, Hongye Fang, Tianyi Wang, Junchi Yan
arXiv:2603. 11211v3 Announce Type: replace-cross Abstract: Incremental Learning (IL) aims to learn new tasks while preserving previously acquired knowledge.
By Haihua Luo, Xuming Ran, Jiangrong Shen, Timo H\"am\"al\"ainen, Zhonghua Chen, Qi Xu, Fengyu Cong
arXiv:2606.21562v2 Announce Type: replace
Abstract: Transformers are AI's workhorse but their computational cost becomes prohibitive when processing long sequences. We target long-horizon streaming v...
By Philippe Weinzaepfel, Christian Wolf, Mert B\"ulent Sariyildiz, Guillaume Bono, Gianluca Monaci
MLLMCLIP introduces a heterogeneous distillation framework that transfers multimodal knowledge from a generative Multimodal Large Language Model (MLLM) teacher directly into a discriminative CLIP student, eliminating the need for synthetic hard negatives. The method uses an attention-based per-layer token selection and a CKA-based distillation loss to bridge architectural differences between the two models. As a result, MLLMCLIP achieves state‑of‑the‑art compositional accuracy and improves zero‑shot classification and image‑text retrieval performance.
By Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao, Hiromi Wakaki, Junmo Kim, Yuki Mitsufuji
The paper introduces FPCD, a framework that stabilizes the modality gap in vision‑language models during continual learning. It combines Fast Null‑space Projection, which projects gradients onto approximate null spaces to reduce interference, with Contrastive Distillation that aligns image‑to‑text and text‑to‑image similarity distributions using a small reference set. On the 11‑task MTIL benchmark, FPCD achieves strong performance, maintaining stable modality gaps and superior zero‑shot retrieval on COCO Captions and Flickr30k.
By Tiantian Peng, Yuyang Liu, Shuo Yang, Qiuhe Hong, YongHong Tian
The paper introduces reViT, a recurrent vision transformer that uses a single Transformer block repeatedly to achieve the accuracy of a full-depth encoder while keeping inference FLOPs comparable. It achieves depth‑specific transformations by expressing each block’s feed‑forward network as a convex combination of a small shared expert bank, with a continuous depth coordinate guiding the mixture. Experiments on ImageNet‑1k and DINOv2 distillation show that reViT matches or surpasses state‑of‑the‑art models, reduces stored parameters by about 70%, and supports elastic‑depth deployment from a single checkpoint.
By Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
arXiv:2511. 11421v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) aims to continually learn new categories without forgetting previously acquired knowledge.
By Lan Li, Tao Hu, Da-Wei Zhou, Jia-Qi Yang, Han-Jia Ye, De-Chuan Zhan
ProCLIP introduces a curriculum‑learning framework that progressively aligns a CLIP image encoder with an LLM‑based text embedder. It first distills CLIP’s text encoder knowledge into the LLM embedder, then fine‑tunes the image encoder via contrastive learning with self‑distillation, instance semantic alignment, and embedding structure alignment losses. The approach addresses CLIP’s token limit, lack of multilingual support, and preserves pre‑trained vision‑language alignment.
By Xiaoxing Hu, Kaicheng Yang, Ziyang Gong, Qi Ming, Zonghao Guo, Yu Tian, Xiang An, Ziyong Feng, Xue Yang
The paper introduces Bidirectional Reciprocal Learning (BRL), a parameter‑efficient fine‑tuning framework for referring image segmentation that operates on frozen vision foundation models. BRL employs two lightweight adapters—Reciprocal Attention Adapter (RAA) for token‑level cross‑modal attention and Reciprocal Gate Adapter (RGA) for channel‑level gating—to enable hierarchical, bidirectional information flow between vision and language. Experiments on RefCOCO, RefCOCO+, and RefCOCOg show that BRL outperforms existing methods while updating fewer than 0.5% of backbone parameters.
By Xiaoqiang Lu, Licheng Jiao, Lingling Li, Yuting Yang, Long Sun, Wenping Ma, Xu Liu, Fang Liu
The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
By Zidan Wang, Yaqian Li, Xiaokai Zhang, Kaiwen Long, Kun He, Hanpeng Liu
arXiv:2609.24510v2 Announce Type: replace
Abstract: Recent advances in vision foundation models (VFMs) have shown remarkable capabilities across diverse unimodal visual tasks. However, adapting VFMs...
By Xiaoqiang Lu, Licheng Jiao, Lingling Li, Yuting Yang, Long Sun, Wenping Ma, Xu Liu, Fang Liu