The paper introduces a new complexity measure, based on a restricted class of "layered zipline programs," which is weaker than the previously defined Arithmetic Repetition Complexity (ARC). This measure allows for a prediction algorithm that operates in quasilinear time and polylogarithmic space on highly-structured sequences, offering a more efficient solution at the cost of reduced expressivity compared to ARC. The work highlights a tradeoff between algorithmic efficiency and the breadth of sequences that can be effectively predicted.
By Vanessa Kosoy
The paper investigates integer‑sequence benchmarks from the OEIS by applying a two‑part minimum description length (MDL) learner that searches for P‑recursive recurrences. It finds that MDL difficulty correlates with a combinatorial parameter count, that most sequences fit a recurrence on a prefix but not at full length (the “wilderness” regime), and that language models do not hallucinate in the wilderness but instead hedge, showing that memorisation dominates perceived competence. The study provides a cheap, contamination‑free difficulty signal for OEIS‑derived benchmarks.
By Sabilashan Ganeshan
The article presents a mathematical framework for large language models (LLMs), detailing how text sequences are encoded into tokens, how next‑token prediction architectures are defined, and how these models are trained and deployed for tasks such as summarization, recommendation, software writing, and quantitative problem solving. It emphasizes that the framework relies on basic concepts from information theory, probability, and optimization, yet captures the complex algorithmic structure responsible for LLMs’ empirical successes. The authors argue that this formalism enables the study of accuracy, efficiency, and robustness, and points toward new methodological developments.
By Ricardo Baptista, Andrew Stuart, Son Tran
arXiv:2607. 12279v1 Announce Type: cross Abstract: Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table.
By Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen
Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires tracking how many tokens remain before a target.
arXiv:2607. 23361v1 Announce Type: cross Abstract: Language generation in the limit is an elegant model introduced by Kleinberg and Mullainathan [KM24] to formally study language generation by an algorithm that learns solely based on example strings.
By Debmalya Panigrahi, Fan Wei, Ian Zhang