← Back to all news
arXiv AI By Melisa Civeleko\u{g}lu, Isabeau Pr\'emont-Schwarz

TPR-Attention for Combinatorial Generalization

Read the original on arXiv AI →

The paper introduces TPR-Attention, an attention mechanism that operates over tensor‑product representations to embed structured inductive bias into deep learning models. Experiments on compositional tasks demonstrate that TPR‑Attention outperforms existing architectural components in achieving combinatorial generalization. The results suggest that incorporating explicit compositional structure into neural attention can improve systematic generalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 10

A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition

The paper presents a necessary and sufficient condition for provable compositional generalization in neural networks, identifying two key principles: structural alignment and unambiguous minimized representations. It rigorously proves this condition, verifies it in Lean 4, and demonstrates its applicability in few-shot settings, including the SCAN jump task. The authors also outline an algorithmic approach and illustrate it with a minimal example, all derived purely from mathematical analysis without empirical validation.

By Yuanpeng Li
safety
More like this →
arXiv Machine Learning
Oct 22

Memorizing Long-tail Data Can Help Generalization Through Composition

The paper investigates how memorizing long‑tail data can aid generalization when combined with a model’s ability to compose features. It demonstrates theoretically that, in a linear setting, memorization plus composition enables correct predictions on rare test examples that involve unseen combinations of long‑tail features. Experiments with neural networks on simple datasets confirm that this effect extends beyond linear models and that a model’s compositional capability depends on its architecture.

By Mo Zhou, Haoyang Ma, Rong Ge
More like this →