The paper investigates how the choice of retrieval encoder affects cache‑based test‑time adaptation for CLIP. By keeping the memory fixed and varying the retrieval space across sixteen encoders, the authors show that retrieval space can dramatically alter performance, with gains ranging from +0.44 to +19.7 points on ImageNet‑A. They introduce MARC, a training‑free system that pairs frozen CLIP with DINOv2‑B for retrieval, achieving superior out‑of‑distribution accuracy and efficiency compared to prior methods.
Cache-based test-time adaptation improves CLIP predictions by storing and retrieving examples from the target stream while keeping the model frozen. However, existing methods largely treat the feature...
arXiv:2607.28571v2 Announce Type: replace Abstract: Operational Earth observation increasingly calls for answering queries such as ``find the image pairs where a new building appeared.'' This means s...
Frozen visual foundation models provide transferable features for visual place recognition, but fixed aggregation can suppress useful distinctions in new environments. We introduce TFA, a reliability-...
The paper introduces TFA, a training‑free aggregation technique that calibrates frozen visual foundation models for visual place recognition. TFA uses cross‑codebook agreement, retrieval coverage, and spectral statistics to adjust residual assignment, spectral shaping, and global‑feature fusion without requiring place labels or task‑specific weights. Experiments with a DINOv2‑B backbone show significant Recall@1 gains over existing training‑free methods across multiple benchmarks, demonstrating that reliability‑guided aggregation can unlock additional retrieval performance from frozen representations.
arXiv:2606. 01790v1 Announce Type: cross Abstract: Vision-language-model-based graphical user interface (GUI) agents have shown broad automation capabilities, yet deployment is bottlenecked by a key-value (KV) cache that grows linearly with interaction steps.
The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
The paper introduces Iris, a framework that extends typed decision‑making models to multimodal settings where candidates can be images, text, or hybrids. Iris preserves key properties such as candidate‑order invariance, scalability, and cache reuse, and achieves high accuracy on a visual‑candidate benchmark while reducing latency through efficient encoding strategies. The authors also demonstrate significant speedups on GPU systems and outperform larger baseline models on MetaEncoder benchmarks.
CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.
arXiv:2606. 31145v2 Announce Type: replace Abstract: Large language models increasingly operate over long contexts, where the KV cache grows linearly with sequence length and dominates memory during decoding.
arXiv:2608. 13144v1 Announce Type: cross Abstract: As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has become a central concern.
The paper introduces Rate‑Controlled Refresh (RCR), a method for re‑selecting key‑value (KV) cache entries during long‑form multimodal generation. RCR lets attention drift determine when and where to refresh, while a per‑sample controller limits how often this occurs, and a low‑rank drift detector keeps the process GPU‑resident. Across multiple models and tasks, RCR outperforms the per‑step retrieval method InfiniGen by up to 3.8 ROUGE‑L points and remains close to exact per‑step re‑selection, with reduced data movement and a smaller GPU‑resident detector.
arXiv:2609.36722v1 Announce Type: new Abstract: Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encodin...