The paper introduces a formal definition of compositional structure based on algorithmic information theory, stating that such structure emerges when combined parts reveal shared patterns not evident when modeled separately. It applies this definition across scales to identify where compositionality arises and then inverts the framework to generate new compositional objects by optimizing over strings and images. The resulting artifacts display clear hierarchical reuse and part‑whole organization, indicating that the definition captures genuine compositionality rather than generic complexity or regularity.
arXiv:2606. 15386v1 Announce Type: new Abstract: Open-ended intelligence is the capacity to adapt to novel problems and environments that are substantially different from those in training.
arXiv:2607. 12634v1 Announce Type: new Abstract: This paper proposes a theoretical framework for understanding intelligence as a process of atomic compression and compositional reuse.
The paper presents a necessary and sufficient condition for provable compositional generalization in neural networks, identifying two key principles: structural alignment and unambiguous minimized representations. It rigorously proves this condition, verifies it in Lean 4, and demonstrates its applicability in few-shot settings, including the SCAN jump task. The authors also outline an algorithmic approach and illustrate it with a minimal example, all derived purely from mathematical analysis without empirical validation.
This paper proposes a theoretical framework for understanding intelligence as a process of atomic compression and compositional reuse. We argue that cognitive, biological, computational, and organizational systems achieve scalable intelligence by decomposing complex phenomena into reusable atomic units that can be recombined into higher-order structures.
arXiv:2605. 08934v2 Announce Type: replace Abstract: Mechanistic interpretability aims to explain neural model behaviour by reverse-engineering learned computational structure into human-understandable components.
arXiv:2505.09716v4 Announce Type: replace Abstract: Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and...
arXiv:2602. 24264v2 Announce Type: replace-cross Abstract: Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems.
arXiv:2506. 20699v2 Announce Type: replace Abstract: Learning in non-stationary and multi-context environments requires more than ordinary within-task generalization.
The paper investigates how ESMFold’s folding trunk transforms amino‑acid sequences into 3D protein structures by using sparse autoencoders (SAEs) to uncover interpretable features. It first analyzes representation distributions across modules and recycles to identify distinct spatial computations, then shows that single representations carry inherited semantics while pair representations compute pairwise geometry. Finally, the authors use a reasoning language model and gradient‑based attribution to trace concept composition across blocks, verifying alignment with known biochemistry and demonstrating that the trained sparse dictionaries can predict the model’s fold quality, thereby turning the black‑box trunk into an interpretable, scientifically verifiable circuit.
arXiv:2503. 10973v2 Announce Type: replace Abstract: Cognition swiftly breaks high-dimensional sensory streams into familiar parts and uncovers their relations.
arXiv:2608. 06956v1 Announce Type: new Abstract: Generative models for matter are often evaluated as samplers over output representations, and their latent spaces are commonly used as proxies for navigating chemical space.
arXiv:2505.09716v3 Announce Type: replace-cross Abstract: Out-of-distribution (OOD) generalisation is considered a hallmark of human and animal intelligence. To achieve OOD through composition, a sys...