arXiv Machine Learning
Jul 14
Globally Consistent Coloring Schemes for Language Identification
arXiv:2607. 11606v1 Announce Type: cross Abstract: We study how little extra information is needed to make adversarial language learning possible.
Language Identification with Succinct Machine-Independent Traces
arXiv:2607. 12443v1 Announce Type: cross Abstract: Motivated by the power of large language models, there has been renewed interest in the Gold-Angluin model of language identification in the limit, with an eye toward variants of the model that might overcome the negative results for its original formulation.
Related stories
arXiv Machine Learning
Jun 29
Safe Language Generation in the Limit
arXiv:2601. 08648v2 Announce Type: replace-cross Abstract: Recent results in learning a language in the limit have shown that, although language identification is impossible, language generation is tractable.
arXiv Machine Learning
Jun 25
Space-Efficient Language Generation in the Limit
arXiv:2606. 25777v1 Announce Type: cross Abstract: We initiate a resource-aware theory of \textit{language generation in the limit} under the minimal constraint of space efficiency.
arXiv Machine Learning
Jul 28
Hallucination Rates in Language Generation
arXiv:2607. 23361v1 Announce Type: cross Abstract: Language generation in the limit is an elegant model introduced by Kleinberg and Mullainathan [KM24] to formally study language generation by an algorithm that learns solely based on example strings.
arXiv Machine Learning
Aug 4
Dense Language Generation Made Simple: Deterministic, Randomized, and Multi-Order Algorithms
arXiv:2608. 01320v1 Announce Type: cross Abstract: Language generation in the limit is a theoretical framework for studying how a generator can learn to produce new valid strings from a stream of positive examples.
arXiv AI
Aug 12
On Solomonoff Induction in Large Language Models and the Limits of Self-Improving: The Singularity Is Not Near Without Symbolic Model Synthesis
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
arXiv Machine Learning
Aug 4
Automatic Annotation of Ancient Greek Vowel Length
arXiv:2608. 01935v1 Announce Type: cross Abstract: Prior work in Ancient Greek NLP relies on corpora that do not disambiguate the phonemic vowel length of alpha, iota, and ypsilon, together known as the dichrona.
arXiv Machine Learning
1d ago
Language models suffer from a curse of ambiguity
arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.
arXiv AI
Aug 6
Protoreasoning in Tiny Transformers
arXiv:2608. 04980v1 Announce Type: cross Abstract: We show that tiny transformers can profitably employ a simple form of Chain of Thought, which we call protoreasoning, allowing us to study step-by-step reasoning on ~1M-parameter models and opening up opportunities for much more detailed experimentation and analysis than is feasible for larger models.
arXiv Machine Learning
Jun 25
Weave of Formal Thought
arXiv:2606. 25987v1 Announce Type: cross Abstract: Large language models (LLMs) attain remarkable surface fluency on code, yet they neither formally guarantee the syntactic validity of their output nor leverage the hierarchical structure defining the target language.
arXiv AI
Jul 22
From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar
arXiv:2607. 18961v1 Announce Type: new Abstract: Large language models (LLMs) generate fluent text by incrementally predicting the next token from a prefix.
arXiv Machine Learning
Jul 8
Boosting with List-Decodable Codes
arXiv:2607. 05791v1 Announce Type: cross Abstract: Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989).