arXiv Machine Learning
Jul 14
Globally Consistent Coloring Schemes for Language Identification
arXiv:2607. 11606v1 Announce Type: cross Abstract: We study how little extra information is needed to make adversarial language learning possible.
Language Identification with Succinct Machine-Independent Traces
arXiv:2607. 12443v1 Announce Type: cross Abstract: Motivated by the power of large language models, there has been renewed interest in the Gold-Angluin model of language identification in the limit, with an eye toward variants of the model that might overcome the negative results for its original formulation.
Related stories
arXiv Machine Learning
Jun 29
Safe Language Generation in the Limit
arXiv:2601. 08648v2 Announce Type: replace-cross Abstract: Recent results in learning a language in the limit have shown that, although language identification is impossible, language generation is tractable.
arXiv Machine Learning
Jun 25
Space-Efficient Language Generation in the Limit
arXiv:2606. 25777v1 Announce Type: cross Abstract: We initiate a resource-aware theory of \textit{language generation in the limit} under the minimal constraint of space efficiency.
arXiv Machine Learning
Jul 28
Hallucination Rates in Language Generation
arXiv:2607. 23361v1 Announce Type: cross Abstract: Language generation in the limit is an elegant model introduced by Kleinberg and Mullainathan [KM24] to formally study language generation by an algorithm that learns solely based on example strings.
arXiv Machine Learning
Aug 4
Dense Language Generation Made Simple: Deterministic, Randomized, and Multi-Order Algorithms
arXiv:2608. 01320v1 Announce Type: cross Abstract: Language generation in the limit is a theoretical framework for studying how a generator can learn to produce new valid strings from a stream of positive examples.
arXiv AI
Sep 21
Large Language Models As Shannon Lossy Compressors Not Solomonoff Induction Estimators: The Singularity Is Not Near Without Symbolic Model Synthesis
The paper argues that Large Language Models (LLMs) do not function as Solomonoff induction estimators because their training objectives—cross‑entropy, negative log‑likelihood, and next‑token prediction—optimize fit to a supplied conditional distribution rather than a program‑weighted universal mixture. It further contends that additional computation alone does not transform these models into optimal predictors without external hyper‑parameter or architectural changes. The authors suggest that neurosymbolic machine learning, exemplified by models such as Fable and Astra, represents a shift toward symbolic model synthesis, moving beyond purely statistical LLMs.
arXiv AI
Aug 12
On Solomonoff Induction in Large Language Models and the Limits of Self-Improving: The Singularity Is Not Near Without Symbolic Model Synthesis
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
arXiv Machine Learning
Aug 4
Automatic Annotation of Ancient Greek Vowel Length
arXiv:2608. 01935v1 Announce Type: cross Abstract: Prior work in Ancient Greek NLP relies on corpora that do not disambiguate the phonemic vowel length of alpha, iota, and ypsilon, together known as the dichrona.
arXiv AI
Sep 15
Large Language Models As Shannon Lossy Compressors Not Solomonoff Induction Estimators: The Singularity Is Not Near Without Symbolic Model Synthesis in Program Space
arXiv:2601.05280v4 Announce Type: replace-cross Abstract: On the one hand, the question of whether Large Language Models (LLMs) are Solomonoff induction estimators has become an explicit question at...
arXiv Machine Learning
Aug 18
Language models suffer from a curse of ambiguity
arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.
arXiv Computation and Language
Sep 7
Improving Language Identification for Code-Switched Utterances with Integer Linear Programming
The paper addresses the challenge of identifying code‑switched utterances in language identification systems. It revisits the MaskLID approach, highlighting its overreliance on word‑level language association scores, and reformulates its optimization as an Integer Linear Program to incorporate clear, interpretable constraints. These enhancements significantly improve performance across ten diverse languages on code‑switching benchmarks, with the authors releasing code and data for reproducibility.
arXiv AI
Aug 6
Protoreasoning in Tiny Transformers
arXiv:2608. 04980v1 Announce Type: cross Abstract: We show that tiny transformers can profitably employ a simple form of Chain of Thought, which we call protoreasoning, allowing us to study step-by-step reasoning on ~1M-parameter models and opening up opportunities for much more detailed experimentation and analysis than is feasible for larger models.