arXiv:2607. 09794v1 Announce Type: new Abstract: Context learning is an emerging inference-time task where LLMs must learn and apply novel, task-specific knowledge from intricate contexts absent from pre-training; even frontier models score under 24% task success.
arXiv:2607. 20064v1 Announce Type: new Abstract: Long-horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents.
arXiv:2609.36788v1 Announce Type: new Abstract: Incorporating rich task-relevant context, such as domain knowledge and external observations, is a key capability yet remains challenging for Bayesian...
arXiv:2604. 27660v3 Announce Type: replace Abstract: Many real-world tasks require language models (LMs) to reason over complex contexts that exceed their parametric knowledge.
The paper introduces Context Language Models (CLMs), which treat context as a mutable file that the model can update freely, enabling the model to learn what information to retain. CLMs built zero‑shot from existing models outperform state‑of‑the‑art context‑management methods on several benchmarks, achieving higher accuracy with fewer FLOPs. The authors also demonstrate that CLMs can be steered via natural‑language instructions and online reinforcement learning, and they propose a suffix‑cache reuse strategy that further reduces server‑side compute.
arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...
arXiv:2605. 21850v2 Announce Type: replace-cross Abstract: Recent development of agents has renewed demand for long-context reasoning capacity of LLMs.
We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted upd...
arXiv:2604.23069v2 Announce Type: replace Abstract: Large language model (LLM) agents often struggle in long-context interactions. As the history grows, sliding windows and prompt compression may dro...
The paper introduces SFT-as-context, a training‑free method that lets a parent large language model (LLM) use the response of a supervised fine‑tuned (SFT) model as context to answer queries. This approach preserves the parent model’s general capabilities while acquiring the SFT model’s specialized skills, achieving performance close to the SFT model on fine‑tuned benchmarks and near the parent model on general tasks. Experiments across 19 parent‑SFT pairs and 11 benchmarks show minimal gaps in performance, and the method can solve problems requiring both specialized and general knowledge, even when neither model alone succeeds.
arXiv:2607. 06160v1 Announce Type: cross Abstract: Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage, insufficient instruction difficulty, and a lack of faithfulness supervision.
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.