arXiv Statistics ML
Oct 26

Empirical concentration for chains with infinite memory

The paper presents empirical concentration inequalities for stationary chains with infinite memory, extending previous results that were limited to Markov chains. It establishes a bounded‑differences inequality under a summable Wasserstein contraction condition and shows how to estimate the necessary dependence coefficients from a single trajectory, using transition counts for finite state spaces or kernel density derivatives for continuous spaces. The authors provide general error bounds that reflect the memory tail and demonstrate that, after applying a fixed safety factor, the resulting empirical bounds are asymptotically as tight as the theoretical ones, while remaining fully adaptive without requiring knowledge of the underlying constants.

By Pierre Alquier, Paul Doukhan
arXiv Machine Learning
Jul 28

Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions

arXiv:2506. 05678v3 Announce Type: replace Abstract: The evolution of sequence modeling architectures, from recurrent neural networks and convolutional models to Transformers and structured state-space models, reflects ongoing efforts to address the diverse temporal dependencies inherent in sequential data.

By Haotian Jiang, Zeyu Bao, Shida Wang, Qianxiao Li