arXiv:2606. 01532v1 Announce Type: new Abstract: Positional encoding (PE) is widely viewed as necessary for transformers to process ordered sequences: without them, the next-token map appears permutation-invariant in its context tokens.
By Qian Li, Xinyu Mao, Shang-Hua Teng
arXiv:2604. 25800v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness.
By Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn
arXiv:2607. 17710v1 Announce Type: new Abstract: Large Language Models (LLMs) have had a remarkable impact across many areas of machine learning.
By Ehsan Futuhi, Nathan R. Sturtevant
arXiv:2607. 26988v1 Announce Type: cross Abstract: What types of decision problems can a causally masked, finite-precision transformer solve for inputs of arbitrary length?
By Franz Nowak, Ryan Cotterell, Reda Boumasmoud
arXiv:2606. 19697v1 Announce Type: cross Abstract: The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation.
By Yanhong Li, Anej Svete, Ashish Sabharwal, William Merrill
arXiv:2608. 11909v1 Announce Type: new Abstract: Two accounts recur in explanations of the success of rotary position embeddings (RoPE).
By Selim Jerad, Anej Svete, Jiaoda Li, Ryan Cotterell