arXiv AI
Dissociating performance from compositional feature learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jun 22
Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv AI
Sep 10
A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition
The paper presents a necessary and sufficient condition for provable compositional generalization in neural networks, identifying two key principles: structural alignment and unambiguous minimized representations. It rigorously proves this condition, verifies it in Lean 4, and demonstrates its applicability in few-shot settings, including the SCAN jump task. The authors also outline an algorithmic approach and illustrate it with a minimal example, all derived purely from mathematical analysis without empirical validation.
arXiv Machine Learning
Jul 7
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models
arXiv:2602. 24264v2 Announce Type: replace-cross Abstract: Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems.
arXiv AI
Jun 9
Muon Learns More Robust and Transferable Features than Adam
arXiv:2606. 09658v1 Announce Type: cross Abstract: Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers.
arXiv AI
Sep 10
Instance-wise Linearization of Neural Network for Model Interpretation
arXiv:2310.16295v2 Announce Type: replace-cross Abstract: Neural network have achieved remarkable successes in many scientific fields. However, the interpretability of the neural network model is sti...
arXiv AI
Sep 11
Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search
The paper introduces a framework for learning feature transformations on tabular data that addresses three limitations of prior generative methods: neglect of hierarchical feature relationships, bias from order-sensitive embeddings, and reliance on gradient-based search. It combines a permutation‑invariant hierarchical module using self‑attention pooling to capture interactions across features, operations, and abstraction levels, with a policy‑guided multi‑objective reinforcement learning strategy that starts from strong seeds and optimizes both predictive accuracy and transformation efficiency. Experiments on diverse tabular benchmarks show the approach outperforms strong baselines, and the authors provide public code and data.