← Back to all news
arXiv Machine Learning By Syed Ibrahim Omer, Ginny Y. Wong. Xiangyu Zhao

MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models

Read the original on arXiv Machine Learning →

MaRK (Markov‑adapted Recurrent Kernels) is a framework that conditions pre‑trained State Space Models (SSMs) by mapping context vectors into bounded modulations of the SSM’s recurrence, read‑in, read‑out, skip, and discretization parameters. By treating the SSM as a linear‑parameter‑varying system, MaRK generates a context‑indexed family of Markov parameter sequences, enabling each diffusion timestep to reshape the model’s input‑output memory kernel. Implemented on a frozen 111 M‑parameter Hydra SSM, MaRK uses low‑rank adapters (Hypernet, Chebyshev polynomial, or Discrete Cosine Transform) to achieve parameter‑efficient fine‑tuning with only 6.3–11 M trainable auxiliary parameters, while providing an analytic stability certificate and demonstrating stable, timestep‑conditioned memory profiles in experiments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 11

Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba

arXiv:2503. 18970v4 Announce Type: replace Abstract: Structured State Space Models (SSMs) have become a prominent class of sequence models, developed against two long-standing difficulties: the sequential computation and gradient propagation limits of Recurrent Neural Networks (RNNs), and the quadratic time and memory cost of self-attention in Transformers.

By Shriyank Somvanshi, Md Monzurul Islam, Mahmuda Sultana Mimi, Sazzad Bin Bashar Polock, Gaurab Chhetri, Anandi Dutta, Amir Rafe, Subasish Das
llms
More like this →
arXiv AI
Sep 3

CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models

The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.

By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
llmsroboticsbenchmarks
More like this →
arXiv Machine Learning
Jul 28

Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions

arXiv:2506. 05678v3 Announce Type: replace Abstract: The evolution of sequence modeling architectures, from recurrent neural networks and convolutional models to Transformers and structured state-space models, reflects ongoing efforts to address the diverse temporal dependencies inherent in sequential data.

By Haotian Jiang, Zeyu Bao, Shida Wang, Qianxiao Li
llmsefficiencybenchmarks
More like this →