← Back to all news
arXiv Machine Learning By Richard F. M. Lim, Ruriko Yoshida

On the Representational Geometry of Dynamic Programs

Read the original on arXiv Machine Learning →

The paper examines why standard neural architectures struggle to generalize to longer inputs when solving dynamic programming (DP) problems. It shows that every finite min-plus DP can be represented as a shortest‑path problem on a directed acyclic graph, equivalently as a tropical polynomial whose extended Newton polyhedron captures the decision boundary of the winning path. The authors prove that the graph, polynomial, and polyhedron descriptions form isomorphic semirings at both the formal polynomial and computed function levels, and they demonstrate that the natural dimensionality‑reduction operations in this semiring are neither injective nor closed, revealing structural limitations that hinder length‑generalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 18

A Computational Tropical Geometry Framework for Neural Networks

The paper introduces a computational tropical geometry framework for symbolically analyzing neural networks with tropical activations. It presents an algorithm that computes the network’s linear regions as explicit unions of polyhedra, proves its correctness, and connects the number of linear regions to the monomials in the tropical expression. The authors also define the Hoffman constant to bound distances to the farthest linear region and release the open‑source Julia library TropicalNN.jl to implement these tools, demonstrating their use on proof‑of‑concept examples.

By Paul Lezeau, Thomas Walker, Yueqi Cao, Shiv Bhatia, Anthea Monod
More like this →
arXiv AI
Aug 14

Algebraic Decomposition Theory for Transformer Length Generalization

arXiv:2608. 13433v1 Announce Type: cross Abstract: Transformer-based language models are known to sometimes generalize to sequences longer than seen during training, but we lack a precise characterization of which tasks admit length generalization.

By Andy Yang, Blerta Veseli, Corentin Barloy, Micha\"el Cadilhac, Andreas Krebs, Charles Paperman, Howard Straubing, Michael Hahn
llms
More like this →
Hugging Face Trending Papers
Jul 15

Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at the extreme of this spectrum, when the architecture's expressible function class collapses to a finite-dimensional algebraic variety?

benchmarks
More like this →