arXiv:2601.05280v4 Announce Type: replace-cross
Abstract: On the one hand, the question of whether Large Language Models (LLMs) are Solomonoff induction estimators has become an explicit question at...
By Hector Zenil, Abicumaran Uthamacumaran, Luan Ozelim
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
By Hector Zenil
arXiv:2610.14007v1 Announce Type: cross
Abstract: A familiar line of reasoning runs as follows: language models are flexible approximators trained under a strictly proper loss, so scale brings them c...
By Luan Ozelim, Abicumaran Uthamacumaran, Hector Zenil
The paper investigates how well practical machine‑learning algorithms approximate Solomonoff’s universal predictor when trained by minimizing log‑loss over diverse datasets. It establishes tight bounds on the worst‑case increase in prediction error caused by approximating the Bayesian mixture predictor, showing that while the excess error can be large in low‑prior environments, it remains bounded on average and under distribution shifts. The authors also extend their analysis to finite‑sample settings relevant to neural networks and Transformers.
By Qian Li, Shang-Hua Teng
The paper investigates the relationship between large language models (LLMs) and compression, showing that LLMs, modeled as next‑token predictors, are equivalent to monotone (order‑preserving) compression algorithms with an error gap of at most 2. It further proves that this monotonicity is necessary for the equivalence only if cryptographic one‑way functions exist, and derives a cryptographic corollary linking next‑bit pseudoentropy to monotone incompressibility.
By Noam Mazor, Andrew Morgan, Rafael Pass
arXiv:2607. 28667v1 Announce Type: cross Abstract: Recent work has shown that classifying large language models (LLMs)' responses can be distinguished by modeling token embeddings as trajectories of a black-box dynamical system (DS) and comparing prediction residuals of two DSs.
By Mohamed Akrout, Dan Wilson