arXiv Machine Learning July 3, 2026 By Zichao Wei

On the Role of Directionality in Structural Generalization

Read the original on arXiv Machine Learning →

arXiv:2607. 02307v1 Announce Type: cross Abstract: Several SLOG test categories explicitly involve directional distinctions (modifier position shifts, argument extraction positions), yet AM-Parser, the previous SOTA, uses an AM algebra whose operations do not encode direction.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

benchmarks

arXiv AI

Aug 18

Structural Generalization on SLOG without Hand-Written Rules

arXiv:2604. 26157v4 Announce Type: replace-cross Abstract: Structural generalization in semantic parsing requires systems to apply learned compositional rules to novel structural combinations.

By Zichao Wei

llms benchmarks

arXiv AI

Aug 7

Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

arXiv:2608. 06111v1 Announce Type: cross Abstract: Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to \textit{syntactic structure}.

By Haris Riaz, Hyungji Kim, Mihai Surdeanu

llms rag benchmarks

arXiv AI

Jun 18

NeSyCat Torch: A Differentiable Tensor Implementation of Categorical Semantics for Neurosymbolic Learning

arXiv:2606. 19279v1 Announce Type: new Abstract: Neurosymbolic semantics is fragmented: classical, fuzzy, probabilistic and neural systems each define truth by their own inductive rules.

By Daniel Romero Schellhorn, Till Mossakowski, Bj\"orn Gehrke

efficiency

arXiv Machine Learning

Sep 18

Relational Attention for Data-Efficient Language Modeling

Relational BabyLM is a decoder‑only Transformer that replaces standard self‑attention with a Dual Attention Transformer (DAT) to separate object‑level lexical features from structural/relational information. The model incorporates a Next‑Latent Prediction objective to compress history into a dense belief state and introduces a RoPE‑based symbol‑retrieval mechanism. On the BabyLM 2026 challenge, the best model ranks 6th overall and 3rd on the NLP‑task subset, outperforming GPT‑2 on most benchmarks and achieving the highest EWoK score among strict‑track entries.

By Adrian Brasoveanu, Ece Takmaz, Jakub Dotla\v{c}il

llms nlp benchmarks

arXiv AI

Jul 22

From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar

arXiv:2607. 18961v1 Announce Type: new Abstract: Large language models (LLMs) generate fluent text by incrementally predicting the next token from a prefix.

By Remo Pareschi

llms data-engineering

arXiv Machine Learning

Sep 22

Guiding the coarse levels of semantic IDs makes the fine levels learnable

arXiv:2609.22227v1 Announce Type: cross Abstract: Generative retrieval represents each item by a short Semantic ID and casts recommendation as autoregressive generation of that sequence. Because the...

By Bin Wang, Zhengyu Zhang

llms rag nlp reinforcement-learning safety

One email a morning, machine-written

Related stories

Structural Generalization on SLOG without Hand-Written Rules

Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

NeSyCat Torch: A Differentiable Tensor Implementation of Categorical Semantics for Neurosymbolic Learning

Relational Attention for Data-Efficient Language Modeling

From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar

Guiding the coarse levels of semantic IDs makes the fine levels learnable