The paper investigates how access to external tools affects the computational expressivity of finite‑precision recurrent models. It shows that any finite‑state bounded‑interface tool can be internalized with only a logarithmic increase in recurrent bits, while a single unbounded tape with simple read/write/move commands suffices for Turing completeness, enabling a single‑layer selective affine state‑space model to simulate any Turing machine with only logarithmic recurrent bits. The study provides exact separations between tool‑augmented and tool‑free regimes under deterministic semantics.
arXiv:2608. 11066v1 Announce Type: cross Abstract: We prove inference-time quantum coordination advantages for specified AI state-tracking tasks.
The paper proposes a fixed‑width recurrent feedback scheme for Recurrent Tsetlin Machines (RTMs) by folding clause activations with XOR, retaining the folded bits at two time scales, and thresholding them back to binary. This compression reduces 480 clause activations to 96 recurrent bits while achieving comparable accuracy (61.47 ± 6.74% and 62.94 ± 9.92%) on a reproducible Boolean finite‑state‑machine benchmark across 144 runs. The study shows that raw clause feedback offers only marginal accuracy gains but increases recurrent width and execution time significantly, and highlights the importance of no‑feedback controls in sequence model benchmarking.
arXiv:2606. 19697v1 Announce Type: cross Abstract: The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation.
The paper introduces the Language Model Agent Machine (LAM), a framework that quantifies the computational resources consumed by language‑model agents’ harnesses—such as bounded context, memory, tools, and verification. It presents four main theoretical results: (1) communication costs mapped to pebbling lower bounds, (2) memory‑access separations revealing a linear gap between random and non‑speculative access, (3) recomputation bounds for bit‑reversal DAGs that show when storing intermediate state is beneficial, and (4) reliability bounds that yield optimal verification intervals. Experiments on GPT‑6 Astra validate the theory’s predictions for communication, reliability, and trade‑offs in practical tasks.
arXiv:2607. 21000v1 Announce Type: new Abstract: Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones.
Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones. In our diagnostic suite, the strongest efficient baselines tend to solve only one side well.
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.
ReplaySSM is a technique that redefines how state-space models (SSMs) handle their recurrent state by caching SSM inputs instead of the state itself. It unrolls the recurrence into a checkpoint plus a buffer of recent inputs, reducing state memory traffic by about half and enabling efficient speculative rollback. The method also allows separate precision levels for checkpoints, improving output accuracy while accelerating decoding in hybrid models such as Mamba-2 and Gated DeltaNet.
arXiv:2604. 14501v2 Announce Type: replace-cross Abstract: We study how depth, finite precision, state dimension, and chain-of-thought (CoT) affect the expressive power of multi-layer state-space models (SSMs).
The paper introduces baseVultra, a compact recurrent neural network cell that reallocates most of its recurrent capacity to a single dense candidate‑generation matrix while simplifying gate operations to O(H) diagonal interactions. On character‑level language modeling benchmarks such as TinyShakespeare and text8, baseVultra matches or slightly outperforms parameter‑matched LSTM and GRU cells, achieving lower validation loss and competitive bits‑per‑character scores. The authors find that the advantages of this candidate‑focused design are most pronounced in low‑capacity regimes, whereas full‑gate cells remain competitive when hidden size is increased.
arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.
M$^2$RNN introduces matrix‑valued hidden states and expressive non‑linear transitions for recurrent neural networks, addressing the limited expressive power of transformers. The architecture scales efficiently with tensor cores, achieving perfect state‑tracking generalization on unseen sequence lengths and outperforming hybrid Gated DeltaNet models by 0.4–0.5 perplexity points on a 7B MoE model while using three‑times smaller recurrent states. Even a single M$^2$RNN layer in a hybrid architecture yields accuracy gains comparable to full Hybrid M$^2$RNN, and improves long‑context performance on LongBench by up to 8 points over state‑of‑the‑art hybrid linear attention models.