The paper investigates the teacher‑student framework used in Test‑Time Adaptation (TTA) and questions the common practice of updating the teacher via an exponential moving average of the student. The authors demonstrate that error accumulation still occurs, especially over longer sequences, and propose an intransigent teacher that remains fixed. This modification yields significant performance gains across multiple datasets, longer scenarios, and various architectures, including semantic segmentation, while also improving robustness to hyperparameter changes.
By Damian S\'ojka, Marc Masana, Bart{\l}omiej Twardowski, Sebastian Cygert
arXiv:2607. 23735v1 Announce Type: new Abstract: In many real-world scenarios, encountering continual shifts in domain during inference is very common.
By Anurag Roy, Riddhiman Moulick, Vinay Kumar Verma, Saptarshi Ghosh, Abir Das
arXiv:2610.34092v1 Announce Type: new
Abstract: Few-step distillation for autoregressive (AR) video generation commonly first initializes an AR student and then refines it with distribution matching...
By Zixuan Duan, Yabo Chen, Xunzhi Xiang, Xin Zhang, Haibin Huang, Yin Chen, Chi Zhang, Qi Fan
arXiv:2608. 01263v1 Announce Type: new Abstract: On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions at prefixes along those trajectories.
By Leyan Xue, Feng Xiong, Mingjun Ma, Changqing Zhang
Tail Cues, Principal Corrections: Plug-and-Play Rectification for Open-Set Test-Time Adaptation introduces TALOR, a lightweight module that rectifies features for non-stationary target streams by decomposing them into high-gain principal and low-gain tail coordinates. TALOR applies a soft-routed affine correction in the principal subspace, subtracts drift via weighted ridge regression, and reconstructs the feature for the source head. On ImageNet-C with six csOOD datasets, TALOR surpasses the next-best UniEnt by 2.2% H-score, runs 2.5× faster, and uses only 12% of its GPU memory, while also boosting COME to 68.4% H-score with minimal memory overhead.
By Yingkai Yang, Chaoqi Chen, Hui Huang
Open-Set Test-Time Adaptation (OSTTA) operates on non-stationary target streams where covariate and semantic shifts coexist. Existing TTA methods often obtain adaptation signals by updating model para...
Deep neural nets achieve remarkable performance when training and test data share the same distribution, but this assumption frequently breaks in real-world deployment, where data undergoes continual distributional shifts. Continual Test-Time Adaptation (CTTA) addresses this challenge by adapting pretrained models to non-stationary target distributions on-the-fly, without access to source data or labeled targets, while mitigating two critical failure modes: catastrophic forgetting of source knowledge and error accumulation from noisy pseudo-labels over extended time horizons.
The paper introduces UnInfo, a test‑time adaptation method for vision‑language models like CLIP that addresses image corruption—a realistic distribution shift caused by sensor conditions. UnInfo leverages uniformity‑aware confidence maximization, information‑aware loss balancing, and knowledge distillation from an EMA teacher to preserve embedding uniformity and improve zero‑shot classification accuracy. Experiments show that UnInfo outperforms existing TTA methods on corrupted image datasets.
By Kazuki Adachi, Shin'ya Yamaguchi, Tomoki Hamagami
The paper introduces TALON, a Task‑Adaptive LoRA‑Teacher framework for Few‑Shot Class‑Incremental Learning. TALON assigns a dedicated LoRA‑Teacher to each incremental task, then distills the frozen teachers into a single LoRA‑Student via Ensemble Knowledge Transfer, using a semantic‑guided weighting scheme to reduce forgetting and overfitting. Experiments on four FSCIL benchmarks show that TALON matches or surpasses state‑of‑the‑art accuracy while using up to 33× fewer deployment parameters and cutting inference time by 41.7%.
By Hongwei Zhao (School of Computer Science,Engineering, Beihang University), Rui Liu (School of Computer Science,Engineering, Beihang University), Yansong Liu (School of Computer Science,Engineering, Beihang University), Zhiyuan Zou (School of Computer Science,Engineering, Beihang University), Yong Chen (School of Computer Science, Beijing University of Posts,Telecommunications)
arXiv:2607. 15851v1 Announce Type: cross Abstract: A range of methods aim to enhance the performance of vision-language models (VLMs) at test time.
By Jiazhen Huang, Zhiming Liu, Changhu Wang, Wei Ju, Ziyue Qiao, Xiao Luo
arXiv:2606. 14299v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment.
By Jiazhen Huang, Xiao Chen, Zhiming Liu, Yaru Sun, Jingyan Jiang, Zhi Wang
The paper introduces CASTER, a gradient‑free test‑time adaptation method that keeps the model frozen by storing source class statistics in a discriminative subspace and applying an affine transformation estimated from target‑batch moments. CASTER avoids backward passes, optimizer state, and large feature banks, outperforming k‑NN on frozen features in most backbone‑dataset settings while using far less memory. The authors also propose a residual‑to‑margin transportability certificate that flags when affine transport is unreliable, and demonstrate that gating based on this certificate can recover performance losses.
By Salim Khazem, Ibrahim Mohamed Serouis