arXiv AI

Adapting Without Gradients: Affine Statistics Transport and What Its Certificate Can Tell You

The paper introduces CASTER, a gradient‑free test‑time adaptation method that keeps the model frozen by storing source class statistics in a discriminative subspace and applying an affine transformation estimated from target‑batch moments. CASTER avoids backward passes, optimizer state, and large feature banks, outperforming k‑NN on frozen features in most backbone‑dataset settings while using far less memory. The authors also propose a residual‑to‑margin transportability certificate that flags when affine transport is unreliable, and demonstrate that gating based on this certificate can recover performance losses.

arXiv Machine Learning

Sep 3

FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers

FORGE is a forward‑only test‑time adaptation technique designed for integer‑only vision models running on microcontrollers. It restores batch‑normalization statistics after BN folding by re‑normalizing each convolution’s per‑channel output using only forward‑pass estimates, enabling adaptation on deployed, folded integer models. The method achieves accuracy gains comparable to gradient‑based TENT, requires adapting only a few layers, works with single‑sample streaming, and has been validated on an ESP32‑S3 with minimal energy and latency overhead.

By Muhammad Rehan, Haider Ali, Muhammad Ali Munir, Moaz Amjad

efficiency

arXiv Computer Vision

Aug 25

ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization

arXiv:2603.14254v2 Announce Type: replace Abstract: Test-time adaptation (TTA) aims to improve model robustness under distribution shifts by adapting to unlabeled test data, but most existing methods...

By Ronghao Zhang, Shuaicheng Niu, Qi Deng, Yanjie Dong, Jian Chen, Runhao Zeng

efficiency safety

arXiv Machine Learning

Aug 4

Logit-Origin Centering for Singleton Test-Time Adaptation

arXiv:2608. 01074v1 Announce Type: new Abstract: Tabular data is used extensively in many real-world use cases.

By Mayank Sharma, Rohit Kumar Mourya, Pratik Mazumder

llms computer-vision benchmarks

arXiv Machine Learning

Aug 5

NOMADD: Numerical Optimization of Models Adapting to Data Drift

arXiv:2608. 02845v1 Announce Type: new Abstract: Tabular model performance degrades when feature distributions change over time or the relationship between features and outcome variables change over time, known as data drift and concept drift, respectively.

By Swapn Shah, Keith Burghardt

benchmarks

arXiv AI

Aug 11

TLDChoiceNet: Quantitatively Choosing a Transfer Learning Dataset

arXiv:2608. 09091v1 Announce Type: cross Abstract: Transfer learning is particularly useful in settings with limited training data, and within image classification it is common to transfer learn upon massive datasets like ImageNet , CIFAR-100, or COCO .

By Jing Ning, James D. Braza

rag computer-vision fine-tuning

arXiv AI

Oct 1

dattri-LLM: A Unified and Efficient Library for Training Data Attribution at LLM Scale

The paper introduces dattri-LLM, a library designed to make training data attribution (TDA) practical for large language models. It achieves efficiency by using compact gradient representations and a cost‑based routing system, while maintaining compatibility by capturing per‑example gradients from existing training loops without modifications, even in distributed settings. The library also offers extensibility through reusable gradient operations and callbacks, supporting various attribution methods and applications such as online data selection, and demonstrates significant performance gains and scalability up to 110B‑parameter models.

By Shixuan Liu, Tongli Zhou, Junwei Deng, Pingbang Hu, Jiaqi W. Ma

llms efficiency

arXiv Machine Learning

Sep 22

SPeaR: Test-Time Adaptation with Steering Primitives for Realigning Representations

arXiv:2609.24111v1 Announce Type: new Abstract: Test-time adaptation (TTA) addresses distribution shift using only unlabeled test data. Existing methods typically adapt pretrained models by updating...

By Muhammad Sudipto Siam Dip, Ali Etemad

llms

arXiv Machine Learning

Aug 5

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System

arXiv:2602. 06932v5 Announce Type: replace Abstract: Speculative decoding can significantly accelerate LLM serving, yet most deployments today disentangle speculator training from serving, treating speculator training as a standalone offline modeling problem.

By Junxiong Wang, Fengxiang Bie, Jisen Li, Zhongzhu Zhou, Zelei Shao, Yubo Wang, Yinghui Liu, Qingyang Wu, Avner May, Sri Yanamandra, Ce Zhang, Tri Dao, Percy Liang, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu

llms reinforcement-learning

arXiv AI

Oct 1

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

arXiv:2605.07111v3 Announce Type: replace-cross Abstract: Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater represe...

By Haozhan Tang, Xiuqi Zhu, Xinyin Zhang, Boxun Li, Virginia Smith, Kevin Kuo

llms fine-tuning data-engineering

arXiv Machine Learning

Oct 9

Training on the Future: A Delay-Aware Audit of Test-Time Adaptation for Time-Series Forecasting

arXiv:2610. 12232v1 Announce Type: new Abstract: Test-time adaptation (TTA) methods for time-series forecasting update a deployed model, or a small adapter around it, from incoming ground truth.

By Mohamed Readh Fentazi, Mazene Ameur, Adlen Ksentini

benchmarks

arXiv Machine Learning

Oct 7

Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

arXiv:2610.07499v1 Announce Type: new Abstract: Multimodal wearable systems must remain reliable when sensor streams become noisy or unavailable. Existing multimodal test-time adaptation (TTA) method...

By Payal Mohapatra, Yueyuan Sui, Haodong Yang, Benjamin Lundell, Stephen Xia, Qi Zhu

multimodal

arXiv Machine Learning

Aug 20

GEAR: Generative Expansion and Real Anchoring for Two-Stage Distillation of Tabular Foundation Models

GEAR is a two‑stage framework that distills tabular foundation models into lightweight MLP or tree‑based predictors for efficient CPU deployment. In the first stage, synthetic covariates are used as teacher‑query locations to train the student on soft TFM targets, expanding coverage beyond observed rows. The second stage re‑anchors the student to the target distribution using real labels and out‑of‑fold teacher predictions, preventing self‑labeling leakage and improving performance. Experiments on TALENT and TabArena show that GEAR‑distilled MLPs outperform supervised MLPs by up to 2.00 AUC points on binary tasks and 1.35 on multiclass tasks, and also outperform CatBoost, while dramatically reducing inference time and memory usage.

By Qi Qin, Jiajie Zhu, Dali Chen, Yuzhao Zhang, Jia-Xing Han, Yu Su, Peng Zhang, Ying Yan, Yifan Sun

llms efficiency

Related stories