Test time training enhances in-context learning of nonlinear functions

The paper studies how test‑time training (TTT) improves in‑context learning (ICL) for nonlinear models, focusing on single‑index models where features lie in a hidden low‑dimensional subspace. By applying TTT to single‑layer transformers trained with gradient‑based methods, the authors derive an upper bound on prediction risk and show that TTT allows the model to adapt to both feature vectors and link functions that vary across tasks—something ICL alone struggles to achieve. They also provide a convergence rate indicating that predictive error can approach the noise level as context size and network width increase.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms