arXiv Machine Learning
Rethinking Expressivity and Efficiency in Test-Time Training
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Sep 1
Test-Time Training with Next-Token Prediction
arXiv:2606.21803v2 Announce Type: replace Abstract: Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test t...
arXiv AI
Jul 13
Self-Guided Test-Time Training for Long-Context LLMs
arXiv:2607. 09415v1 Announce Type: cross Abstract: Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs.
arXiv AI
Aug 11
Length-MAX Tokenizer for Language Models
arXiv:2511. 20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed to represent text during training and to generate text during inference.
arXiv Machine Learning
Jul 14
Controllably Efficient Language Models
arXiv:2511. 05313v2 Announce Type: replace Abstract: The substantial inference costs of attention in transformers motivated the development of efficient sequence mixers: namely sparse and sliding window attention, convolutions and linear attention.
arXiv Machine Learning
Aug 4
Learning What to Remember: Test-Time Training via Context Distillation
arXiv:2608. 01672v1 Announce Type: cross Abstract: Effective long-context modeling is not merely about retaining more of the past, but about preserving the information that may prove relevant later.
arXiv AI
Jul 24
SR-TTT Does Not Learn Retrieval: A Correction and Mechanistic Post-Mortem of Surprisal-Aware Residual Test-Time Training
arXiv:2603. 06642v2 Announce Type: replace-cross Abstract: Test-Time Training (TTT) language models replace the KV-cache with fast weights updated during inference, achieving O(1) memory but suffering catastrophic failure on exact-recall tasks.