arXiv Machine Learning August 4, 2026 By Zixuan Wang, Xingyu Dang, Rui-Jie Zhu, Zixin Wen, Hengyu Fu, Wenhao Chai, Jason D. Lee

Learning What to Remember: Test-Time Training via Context Distillation

Read the original on arXiv Machine Learning →

arXiv:2608. 01672v1 Announce Type: cross Abstract: Effective long-context modeling is not merely about retaining more of the past, but about preserving the information that may prove relevant later.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 13

Self-Guided Test-Time Training for Long-Context LLMs

arXiv:2607. 09415v1 Announce Type: cross Abstract: Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs.

By Xinyu Zhu, Zhe Xu, Xiaohan Wei, Yunchen Pu, Fei Tian, Chonglin Sun, Kaushik Rangadurai, Hua Zhi, Frank Shyu, Sandeep Pandey, Luke Simon, Yu Meng, Xi Liu

llms benchmarks

arXiv AI

Jul 14

Extending LLM Context via Associative Recurrent Memory

arXiv:2607. 11614v1 Announce Type: cross Abstract: Extending the context length of large language models (LLMs) is critical for many real-world applications, yet standard transformers remain constrained by quadratic compute and linear memory scaling.

By Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov, Yuri Kuratov, Lyudmila Rvanova, Mikhail Katkov, Ilia Sochenkov, Misha Tsodyks, Timothy Baldwin, Mikhail Burtsev, Artem Shelmanov

llms fine-tuning

arXiv AI

Jul 23

LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning

arXiv:2607. 19358v1 Announce Type: new Abstract: Recent advances in long chain-of-thought reasoning models such as DeepSeek-R1 have led to increasingly longer inference context lengths under the test-time scaling paradigm.

By Yu Zhao, Zekun Zhang, Fan Jiang, Bo Zeng, Linlong Xu, Shimin Shan, Yu Liu, Longyue Wang, Weihua Luo

efficiency benchmarks

arXiv AI

Jun 4

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training

arXiv:2506. 05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.

By Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Sarthak Mittal, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pascanu, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento

llms benchmarks

arXiv Computation and Language

Sep 1

Test-Time Training with Next-Token Prediction

arXiv:2606.21803v2 Announce Type: replace Abstract: Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test t...

By Xuan Ouyang, Zefan Cai, Junjie Hu

llms benchmarks

arXiv Machine Learning

Sep 1

PERK: Long-Context Reasoning as Test-Time Learning

arXiv:2507.06415v3 Announce Type: replace-cross Abstract: Long-context reasoning requires accurately identifying relevant information in extensive, noisy input contexts. In this work, we propose PERK...

By Zeming Chen, Angelika Romanou, Gail Weiss, Antoine Bosselut

llms fine-tuning

One email a morning, machine-written

Related stories

Self-Guided Test-Time Training for Long-Context LLMs

Extending LLM Context via Associative Recurrent Memory

LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training

Test-Time Training with Next-Token Prediction

PERK: Long-Context Reasoning as Test-Time Learning