arXiv AI
Jul 14

Agentic Context Learning with Self-Discovered Specification

arXiv:2607. 09794v1 Announce Type: new Abstract: Context learning is an emerging inference-time task where LLMs must learn and apply novel, task-specific knowledge from intricate contexts absent from pre-training; even frontier models score under 24% task success.

By Jike Zhong, Ming Li, Yuxiang Lai, Ziyan Yang, Jingyu Xie, Jihyung Kil, Zheda Mai, Shao-Yuan Lo, Ren Xiang, Konstantinos Psounis, Yuanyuan Lei
arXiv AI
Sep 30

Context Language Models

The paper introduces Context Language Models (CLMs), which treat context as a mutable file that the model can update freely, enabling the model to learn what information to retain. CLMs built zero‑shot from existing models outperform state‑of‑the‑art context‑management methods on several benchmarks, achieving higher accuracy with fewer FLOPs. The authors also demonstrate that CLMs can be steered via natural‑language instructions and online reinforcement learning, and they propose a suffix‑cache reuse strategy that further reduces server‑side compute.

By Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh