← Back to all news
arXiv AI By Yuanpeng Li

A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition

Read the original on arXiv AI →

The paper presents a necessary and sufficient condition for provable compositional generalization in neural networks, identifying two key principles: structural alignment and unambiguous minimized representations. It rigorously proves this condition, verifies it in Lean 4, and demonstrates its applicability in few-shot settings, including the SCAN jump task. The authors also outline an algorithmic approach and illustrate it with a minimal example, all derived purely from mathematical analysis without empirical validation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 16

Defining and Generating Compositional Structure

The paper introduces a formal definition of compositional structure based on algorithmic information theory, stating that such structure emerges when combined parts reveal shared patterns not evident when modeled separately. It applies this definition across scales to identify where compositionality arises and then inverts the framework to generate new compositional objects by optimizing over strings and images. The resulting artifacts display clear hierarchical reuse and part‑whole organization, indicating that the definition captures genuine compositionality rather than generic complexity or regularity.

By Eric Elmoznino, Tejas Kasetty, Blaise Ag\"uera y Arcas, Yoshua Bengio, Guillaume Lajoie
More like this →
arXiv AI
Sep 18

Compositional Reasoning in Language Models under Reinforcement Learning Post-Training

The paper introduces a dependency‑graph framework to formalize compositional reasoning in language models, defining three increasing levels of compositionality. Using data‑structure tasks with deterministic rewards, the authors observe a consistent asymmetry: training on decomposed skills does not reliably transfer to composed tasks, whereas training on composed tasks transfers more readily to decomposed ones. They provide a theoretical explanation for this asymmetry and evaluate its effects under length extrapolation, structural distribution shift, and transfer to unseen skills, concluding with a pilot study on real‑world tool‑calling benchmarks that suggests the phenomenon extends to practical settings.

By Yu He, Yingxi Li, Yifei Wang, Ellen Vitercik
llmsreinforcement-learningbenchmarks
More like this →
arXiv AI
Sep 1

TPR-Attention for Combinatorial Generalization

The paper introduces TPR-Attention, an attention mechanism that operates over tensor‑product representations to embed structured inductive bias into deep learning models. Experiments on compositional tasks demonstrate that TPR‑Attention outperforms existing architectural components in achieving combinatorial generalization. The results suggest that incorporating explicit compositional structure into neural attention can improve systematic generalization.

By Melisa Civeleko\u{g}lu, Isabeau Pr\'emont-Schwarz
safety
More like this →