The paper investigates the thermodynamic cost of inference and learning in physical neural networks. It shows that quasi‑static inference requires no work, while finite‑speed inference incurs work bounded by the Wasserstein‑2 distance between thermal states, roughly $k_B T$ per dimension of the widest layer. Learning, however, has an irreducible cost of a few $k_B T$ per parameter, independent of speed, indicating that memory dominates the thermodynamic price.
By Alexei V. Tkachenko
arXiv:2504.06209v2 Announce Type: replace
Abstract: Predicting future observations plays a central role in machine learning, biology, economics, and many other fields. It lies at the heart of organiz...
By Lukas J. Fiderer, Paul C. Barth, Isaac D. Smith, Hans J. Briegel
The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.
By Kai Niu, Ping Zhang
arXiv:2510. 22052v2 Announce Type: replace Abstract: The field of artificial intelligence (AI) has taken a tight hold on broad aspects of society, industry, business, and governance in ways that dictate the prosperity and might of the world's economies.
By Abhijit Chatterjee, Niraj K. Jha, Jonathan D. Cohen, Thomas L. Griffiths, Hongjing Lu, Diana Marculescu, Ashiqur Rasul, Wenrui Xu, Keshab K. Parhi
arXiv:2601.05280v4 Announce Type: replace-cross
Abstract: On the one hand, the question of whether Large Language Models (LLMs) are Solomonoff induction estimators has become an explicit question at...
By Hector Zenil, Abicumaran Uthamacumaran, Luan Ozelim
The paper argues that Large Language Models (LLMs) do not function as Solomonoff induction estimators because their training objectives—cross‑entropy, negative log‑likelihood, and next‑token prediction—optimize fit to a supplied conditional distribution rather than a program‑weighted universal mixture. It further contends that additional computation alone does not transform these models into optimal predictors without external hyper‑parameter or architectural changes. The authors suggest that neurosymbolic machine learning, exemplified by models such as Fable and Astra, represents a shift toward symbolic model synthesis, moving beyond purely statistical LLMs.
By Hector Zenil, Abicumaran Uthamacumaran, Luan Ozelim
arXiv:2606. 10706v1 Announce Type: cross Abstract: Resource constraints increasingly determine what can be trained, fine-tuned, and deployed in large language models (LLMs), yet efficiency is often studied through isolated techniques rather than as an interacting system of limits.
By Vanessa Schmidt, Huy Hoang Nguyen, C\'edric Jung, Shirin Salehi, Anke Schmeink
The paper investigates how much learned memory is required to leverage additional data in autoregressive prediction models. It introduces a predictive‑energy spectrum that jointly governs data and memory scaling, proving a minimax law that links the number of prediction blocks and the size of the learned state to this spectrum. The authors demonstrate that optimal bit allocation and masked query‑key attention mechanisms realize this law, and they provide experimental evidence across multiple pretrained‑model scales.
By Chiwun Yang, Xiaoyu Li
arXiv:2605. 18909v2 Announce Type: replace Abstract: Any system that models the world under finite representational capacity must compress; any compression entails a prior; and the prior is the system's bias.
By Ahmed Gamal Eldin
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
By Hector Zenil
The paper presents an autonomous agent that designs machine learning algorithms for wireless power control, eliminating manual specification of architecture, loss, and training details. Using an autoresearch protocol, the agent iteratively edits a training script, runs experiments, and evaluates changes against a single metric, ultimately achieving 99.5% of a reference solution with vastly reduced inference cost. The agent’s discovered output parameterization matches the exact max‑min‑optimal allocation at the minimum percentile for all trained weights, demonstrating a principled, scalable approach to a complex, NP‑hard problem.
By Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini, Raviraj S. Adve
arXiv:2606. 20231v1 Announce Type: new Abstract: Can intelligence be measured?
By Ishanu Chattopadhyay