arXiv:2607. 10441v1 Announce Type: cross Abstract: Context engineering decides what information a model carries forward, and current designs meter it in tokens: compressing the past into a bounded recurrent state, keeping a key-value entry for every token, or imposing a fixed budget through a window or eviction rule.
By Siddharth Pal, Viktoria Rojkova
arXiv:2603. 20381v2 Announce Type: replace-cross Abstract: Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions.
By Christopher J. Agostino, Quan Le Thien, Nayan D'Souza, Louis van der Elst
Large Language Models (LLMs) are frequently portrayed as general-purpose solvers capable of solving arbitrary tasks. We argue that this view overlooks a fundamental constraint: language is a compressed and capacity-limited interface for conveying task information.
The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.
By Kai Niu, Ping Zhang
arXiv:2609.27869v1 Announce Type: new
Abstract: Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typ...
By Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Shuo Yang, Edith Cheuk-Han Ngai
The paper introduces Causal Memory Policy (CMP), a framework that identifies the utility of memories in memory‑augmented language models by intervening on retrieval rather than on storage. CMP reserves fixed context slots for memories sampled with known propensities and estimates utility using self‑normalized inverse propensity weighting, providing unbiased estimates and exact variance. Experiments show that CMP improves discrimination between required and non‑required memories and reveals that identified utility alone is insufficient for retention decisions across unseen queries.
By Arman Behnam, Binghui Wang
The paper investigates how language models decide between contextual information and their internal memory when the two conflict. By estimating "authority directions" from agreement prompts and swapping these directions between matched prompts, the authors show that such interventions can reproduce 30–68% of the shift in source choice across Qwen, Llama, and OLMo models. Cross‑task experiments reveal that authority directions learned on one task transfer only modestly (≈9%) to another, indicating that authority computations are largely task‑specific.
By Benjamin Shih, John Winnicki, Arianna Cao
arXiv:2607. 12216v1 Announce Type: cross Abstract: Multi-agent and memory-augmented LLM systems often place coordination content, shared state, prior discussion, tool outputs, summaries, and role instructions, inside the same finite prompt used for the current task.
By Brenda Lelis, Rodrigo Cabral-Carvalho
arXiv:2604. 09670v2 Announce Type: replace-cross Abstract: Intelligent systems must maintain and manipulate task-relevant information online to adapt to dynamic environments and changing goals.
By Hua-Dong Xiong, Li Ji-An, Jiaqi Huang, Robert C. Wilson, Kwonjoon Lee, Xue-Xin Wei
The paper introduces the Language Model Agent Machine (LAM), a framework that quantifies the computational resources consumed by language‑model agents’ harnesses—such as bounded context, memory, tools, and verification. It presents four main theoretical results: (1) communication costs mapped to pebbling lower bounds, (2) memory‑access separations revealing a linear gap between random and non‑speculative access, (3) recomputation bounds for bit‑reversal DAGs that show when storing intermediate state is beneficial, and (4) reliability bounds that yield optimal verification intervals. Experiments on GPT‑6 Astra validate the theory’s predictions for communication, reliability, and trade‑offs in practical tasks.
By Zelin Zhao (Georgia Institute of Technology), Xinyu Guo (Georgia Institute of Technology), Jingyuan Zhang (Georgia Institute of Technology), Yuxuan Zhang (Etude AI), Yongxin Chen (Georgia Institute of Technology)
The paper presents a complete enumeration of all possible representations and binary action rules in a uniform four‑state world, comparing their retained information, payoff (task accuracy), and cost under various tariffs. It shows that a representation with less retained information can sometimes yield higher payoff, while in other cases the more informative representation performs better, and that equal payoff can occur with different information amounts. The study demonstrates how the specific distinctions preserved by a representation, the choice of action rule, and the cost assumptions jointly determine performance, offering a reproducible model for exploring these trade‑offs.
By Aleksandr M. Shtraukh-Adamsky, Vladimir M. Adamsky, Irina G. Voloshina, Artemiy D. Spitsyn
arXiv:2608. 15022v1 Announce Type: new Abstract: Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly.
By Parsa Mazaheri