arXiv:2603. 11784v2 Announce Type: replace Abstract: As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed.
By Giorgio Racca, Michal Valko, Amartya Sanyal
The paper introduces the task of Scientific Claim Unlearning and presents a new benchmark, SciUnlearn, to evaluate it. It highlights that language models trained on static scientific corpora risk disseminating outdated or retracted claims as scientific knowledge evolves. Current machine unlearning methods fail to effectively remove claim-level knowledge, often only suppressing it superficially, underscoring the need for specialized techniques for structured knowledge removal.
By Snigdha Paul, Manasi Patwardhan, Arman Cohan
The paper argues that Large Language Models (LLMs) do not function as Solomonoff induction estimators because their training objectives—cross‑entropy, negative log‑likelihood, and next‑token prediction—optimize fit to a supplied conditional distribution rather than a program‑weighted universal mixture. It further contends that additional computation alone does not transform these models into optimal predictors without external hyper‑parameter or architectural changes. The authors suggest that neurosymbolic machine learning, exemplified by models such as Fable and Astra, represents a shift toward symbolic model synthesis, moving beyond purely statistical LLMs.
By Hector Zenil, Abicumaran Uthamacumaran, Luan Ozelim
arXiv:2604. 27540v2 Announce Type: replace Abstract: Scientific reasoning rarely stops at what is directly observable; it often requires uncovering hidden structure from data.
By Chaemin Jang, Woojin Park, Hyeok Yun, Dongman Lee, Jihee Kim
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
By Hector Zenil
arXiv:2609.37891v1 Announce Type: cross
Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...
By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko