arXiv:2609.38524v1 Announce Type: new Abstract: We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions s...
arXiv:2610. 08816v1 Announce Type: new Abstract: Long-range temporal dependence poses a resource question for sequence models: for a specified predictive-memory law, how much state, context, or dynamical criticality is required in order to forecast accurately?
arXiv:2609.08740v1 Announce Type: new Abstract: In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynam...
arXiv:2608. 13621v1 Announce Type: new Abstract: A hidden Markov model (HMM) combines three roles: inference of a hidden-state belief from observations, propagation through a Markov transition, and emission back to observation space.
The paper investigates whether compression explains why over‑parameterized neural networks generalize, using the minimum description length (MDL) principle. It introduces prequential regret, a computable metric from the training trajectory, and shows that in the nearly interpolating regime it upper‑bounds the description length of the trained parameters, leading to a high‑probability generalization bound in the PAC‑MDL setting. The study also connects MDL with the algorithm‑level information bottleneck by bounding encoder‑complexity correction with MDL parameter description length, and demonstrates that prequential regret correlates strongly (Pearson > 0.9) with generalization error on CIFAR‑10 and CIFAR‑100.
The paper shows that a memory state that reproduces every current prediction can still fail to learn from subsequent observations. It characterizes when a fixed‑dimensional continuous recurrent state exists for exact cumulative learning and attention, proving that for smooth convex residual losses the state exists only when the loss is polynomial. The work further quantifies the dimensionality required for various loss functions, demonstrates unbounded requirements for nonpolynomial losses, and analyzes specific cases such as ridge regression and kernel attention.
A central question in modern learning theory is what notion of compression, if any, explains why over-parameterized neural networks generalize. Existing answers measure complexity at different levels,...
arXiv:2608. 13510v1 Announce Type: cross Abstract: Machine learning procedures are commonly evaluated in terms of predictive accuracy and computational efficiency.
The paper presents empirical concentration inequalities for stationary chains with infinite memory, extending previous results that were limited to Markov chains. It establishes a bounded‑differences inequality under a summable Wasserstein contraction condition and shows how to estimate the necessary dependence coefficients from a single trajectory, using transition counts for finite state spaces or kernel density derivatives for continuous spaces. The authors provide general error bounds that reflect the memory tail and demonstrate that, after applying a fixed safety factor, the resulting empirical bounds are asymptotically as tight as the theoretical ones, while remaining fully adaptive without requiring knowledge of the underlying constants.
arXiv:2608. 07224v1 Announce Type: new Abstract: Motivated by LLMs, which generate outputs by iteratively sampling from next-token distributions, we introduce a PAC-learning model for binary stochastic autoregressive learning.
arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
arXiv:2604.01024v2 Announce Type: replace Abstract: We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to le...