arXiv AI By Song-Ju Kim

Contextual Information Allocation in Shared-State Cognitive Models: An Information-Theoretic Bound

Read the original on arXiv AI →

The paper derives an information‑theoretic bound for a shared‑state cognitive architecture that uses an auxiliary variable to mediate context. It shows that the residual dependence of observable behavior on context, given the shared state, is bounded by the information carried by the auxiliary variable and its conditional entropy. A recognition‑memory example illustrates how to compute and compare this bound across different representational choices, providing a framework for analyzing context‑memory‑control trade‑offs in cognitive models and artificial agents.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jul 14

Context by Distinct Information: An Auditable Dirichlet-Process Working Memory for Long, Redundant Context Streams

arXiv:2607. 10441v1 Announce Type: cross Abstract: Context engineering decides what information a model carries forward, and current designs meter it in tokens: compressing the past into a bounded recurrent state, keeping a key-value entry for every token, or imposing a fixed budget through a window or eviction rule.

By Siddharth Pal, Viktoria Rojkova
arXiv AI
Sep 12

A Mathematical Theory of Pragmatic Information

The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.

By Kai Niu, Ping Zhang
arXiv AI
Oct 2

Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

The paper introduces Causal Memory Policy (CMP), a framework that identifies the utility of memories in memory‑augmented language models by intervening on retrieval rather than on storage. CMP reserves fixed context slots for memories sampled with known propensities and estimates utility using self‑normalized inverse propensity weighting, providing unbiased estimates and exact variance. Experiments show that CMP improves discrimination between required and non‑required memories and reveals that identified utility alone is insufficient for retention decisions across unseen queries.

By Arman Behnam, Binghui Wang