arXiv:2604. 26157v4 Announce Type: replace-cross Abstract: Structural generalization in semantic parsing requires systems to apply learned compositional rules to novel structural combinations.
By Zichao Wei
The paper investigates why Transformers struggle more with structural than lexical compositional generalisation. It argues that this disparity stems from low structural type diversity rather than an inherent limitation of Transformers. By creating linguistically diverse variants of the COGS and SLOG datasets, the authors show that type diversity correlates equally with generalisation in both lexical and structural cases, challenging previous explanations of the difficulty.
By Anssi Moisio, Mathias Creutz, Mikko Kurimo
Coordination, a fundamental linguistic structure, remains a subject of intense debate, and its exact nature continues to elude theoretical linguistics. A common view holds that only same-category constituents can be conjoined, which has been challenged by the many grammatical unlike coordinations found in natural language.
The study investigates how part‑of‑speech (PoS) categories are represented in the latent space of Sparse AutoEncoders (SAEs) applied to language models. Results show that PoS distinctions can be reliably recovered from SAE activations, but the mapping is not one‑to‑one; instead, PoS categories are supported by compact, distributed groups of sparse latents that vary across tags and remain stable on held‑out data. The findings suggest that SAEs encode morpho‑syntactic information in a distributed, category‑dependent manner rather than through isolated grammatical features.
By Alessandro Bondielli, Lucia Passaro, Serena Auriemma, Alessandro Lenci
The paper investigates how verb–noun decomposition, a common strategy for recognizing assembly actions, generalizes to novel combinations of familiar components. Through a systematic study on three datasets (MECCANO, HAViD, and IMPACT), the authors find that while decomposition avoids the zero‑probability ceiling of atomic classifiers, its performance still heavily depends on the co‑occurrence patterns seen during training. The analysis reveals that errors concentrate on the larger‑vocabulary component, that shared‑encoder training can entangle components and worsen generalization, and that these issues stem from primitive support, vocabulary asymmetry, and component entanglement.
By Changyi Li, Yu Xiao
arXiv:2606. 00230v1 Announce Type: new Abstract: Grokking, the phenomenon in which neural networks generalize long after fitting their training data, has been studied in supervised settings on many epochs.
By Sherin Muckatira, Namrata Shivagunde, Vijeta Deshpande, Anna Rumshisky