arXiv Machine Learning
Non-Commutative State Tracking with Input-Dependent Low-Rank Updates in Mamba-3
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 8
A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Tracking
arXiv:2606. 07254v1 Announce Type: new Abstract: State tracking exposes a sharp limitation of sequence models: the relevant signal is often not a summary of observed tokens, but an ordered latent state that evolves through non-commutative transformations.
arXiv Machine Learning
Aug 17
The Expressive Limits of Diagonal SSMs for State-Tracking
arXiv:2603. 01959v2 Announce Type: replace Abstract: State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-parallelizable.
arXiv Machine Learning
Jun 11
Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
arXiv:2602. 10743v2 Announce Type: replace Abstract: State-space language models such as Mamba and gated linear attention (GLA) offer linear-complexity, parallelisable alternatives to transformers, but their linear state updates limit expressivity and robust state tracking.
arXiv AI
Sep 3
CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.
arXiv AI
Jul 14
Looped State-Space Language Models with Adaptive Exit-State Selection
arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.
arXiv Machine Learning
Jul 30
Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
arXiv:2607. 27132v1 Announce Type: new Abstract: An agent acting under partial observability must retain a recursively updateable statistic of history that restores the Markov property, but the smallest such statistic is generally unknown.