arXiv Computer Vision
LAS-CLIP: A Lightweight Adapter Steering Approach for CLIP's Visual Encoder
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 10
Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance
arXiv:2609.05730v1 Announce Type: cross Abstract: Contrastive Language-Image Pretraining (CLIP) is a building block of many machine learning applications. Scaling laws have guided resource allocation...
arXiv AI
5d ago
Rethinking Contrastive Loss in CLIP Post-training: A Complementary Framework with Frozen Text Encoder
The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
arXiv Computer Vision
Sep 22
RefAM: Attention Magnets for Zero-Shot Referral Segmentation
arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
arXiv AI
Oct 5
DAGS: Disentangled Appearance-and-Geometry Steering of a Frozen Image DiT for Temporally Stabilized Generative Rendering
DAGS introduces a lightweight, attention‑free conditioning scheme that disentangles appearance and geometry for a frozen image diffusion transformer (DiT), enabling high‑fidelity, temporally stable renders with independent control. Two small convolutional encoders generate per‑frame conditioning features, which are injected as learned residuals into the image tokens, avoiding the quadratic cost of attention. Coupled with a recurrent lighting stabilizer and a training‑free temporal guidance term, DAGS transforms a per‑frame image model into a streaming renderer that outperforms real‑time denoisers and diffusion renderers in PSNR and temporal stability while requiring far less compute than path tracing.
arXiv AI
Aug 28
LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics
LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.
arXiv Computer Vision
Sep 1
Enhancing Self-Supervised Visual Representation Learning via Low-Rank Adapted LLMs
arXiv:2507.00754v3 Announce Type: replace Abstract: The integration of Large Language Model (LLMs) blocks with Vision Transformers (ViTs) holds immense promise for vision-only tasks by leveraging the...