The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.
Large Language Models (LLMs) are frequently portrayed as general-purpose solvers capable of solving arbitrary tasks. We argue that this view overlooks a fundamental constraint: language is a compressed and capacity-limited interface for conveying task information.
arXiv:2602. 05463v2 Announce Type: replace-cross Abstract: Modern AI systems achieve remarkable capabilities at the cost of substantial energy consumption.
The paper derives an information‑theoretic bound for a shared‑state cognitive architecture that uses an auxiliary variable to mediate context. It shows that the residual dependence of observable behavior on context, given the shared state, is bounded by the information carried by the auxiliary variable and its conditional entropy. A recognition‑memory example illustrates how to compute and compare this bound across different representational choices, providing a framework for analyzing context‑memory‑control trade‑offs in cognitive models and artificial agents.
The paper introduces Embodied Semantic Communication (ESC), a new paradigm that redefines information transmission for autonomous agents by embedding multimodal perceptual states, hardware capabilities, and collaborative intents into unified, action‑oriented semantic representations. ESC enables heterogeneous agents to parse, align, and ground shared information directly into local motor control, addressing the limitations of traditional communication approaches that focus solely on bit delivery or single‑task optimization. The tutorial outlines ESC’s conceptual boundaries, system characteristics, and technical pathways, mapping relevant mathematical tools such as semantic information theory, world models, and multi‑agent decision theory, and concludes with a roadmap of open challenges like semantic reliability, dynamic interaction, and bandwidth‑adaptive transmission.
arXiv:2609.37994v1 Announce Type: cross Abstract: The classical information bottleneck (IB) measures the relevance of a representation $U$ of $X$ to a target $Y$ by $I(U;Y)$, which does not directly...
arXiv:2607. 17232v1 Announce Type: cross Abstract: Classical rate-distortion (RD) theory has long established the fundamental limits of lossy compression by quantifying the minimum number of bits required to represent a source under a prescribed distortion constraint.
arXiv:2607. 15459v1 Announce Type: new Abstract: A trained deep reinforcement learning policy is a black box, and we ask whether it can be made explainable by rewriting it as an executable logic program that reproduces its behaviour and that a person can read, a logic engine can run, and an optimizer can edit.
arXiv:2606. 12502v1 Announce Type: cross Abstract: We propose that value -- the quantity goal-directed agents create, destroy, and exchange -- is a lawful structural quantity in the same category as information.
The paper argues that Large Language Models (LLMs) do not function as Solomonoff induction estimators because their training objectives—cross‑entropy, negative log‑likelihood, and next‑token prediction—optimize fit to a supplied conditional distribution rather than a program‑weighted universal mixture. It further contends that additional computation alone does not transform these models into optimal predictors without external hyper‑parameter or architectural changes. The authors suggest that neurosymbolic machine learning, exemplified by models such as Fable and Astra, represents a shift toward symbolic model synthesis, moving beyond purely statistical LLMs.
arXiv:2604. 10727v2 Announce Type: replace-cross Abstract: Classical information-theoretic learning bounds typically rely on KL mutual information and moment-generating-function (MGF) arguments, which are well matched to bounded or sub-Gaussian losses but can be ineffective when losses or rewards are heavy-tailed.
arXiv:2607. 06570v1 Announce Type: cross Abstract: Value-of-information (VOI) analysis is usually conducted under a single probability measure.
The paper introduces CPR‑IE, a metric that orders intelligent systems by representational economy, predictive quality, and resource burden. It formalizes how raw resource consumption is represented and aggregated, showing that proportional‑increment composition yields logarithmic cumulative burden and context‑independent ratio responses produce power relationships among compression, prediction, and burden. The authors prove properties such as Pareto consistency, unit invariance, and ranking stability, and provide a translog parent model to make interaction restrictions explicit, along with guarantees on ranking and regret bounds.