arXiv:2603. 19954v2 Announce Type: replace Abstract: Transformers have shown inconsistent success in AI planning tasks, and theoretical understanding of when generalization should be expected has been limited.
By Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thi\'ebaux, Michael Hahn
arXiv:2604. 11912v2 Announce Type: replace-cross Abstract: While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks.
By Jianhao Huang, Zhanpeng Zhou, Renqiu Xia, Baharan Mirzasoleiman, Weijie Su, Wei Huang
arXiv:2602. 18948v2 Announce Type: replace Abstract: Transformer models contain substantial internal redundancy arising from coordinate-dependent representations and continuous symmetries, in model space and in head space, respectively.
By J. Fran\c{c}ois, L. Ravera
arXiv:2607. 17624v1 Announce Type: new Abstract: Transformers are remarkably versatile and their design is largely consistent across a variety of applications.
By Damien Teney, Liangze Jiang, Hemanth Saratchandran, Simon Lucey
arXiv:2607. 11875v1 Announce Type: cross Abstract: We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models.
By Tiberiu Musat, Tiago Pimentel, Nicholas Zucchet, Thomas Hofmann
arXiv:2606. 02438v1 Announce Type: new Abstract: Learned heuristics have recently become a competitive alternative to traditional domain-independent heuristics for satisficing planning.
By Windy Phung, Dominik Drexler, Arnaud Lequen, Jendrik Seipp