Memory in Deep Time-Series Models
arXiv:2609.06006v1 Announce Type: cross Abstract: Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state...
arXiv:2609.06006v1 Announce Type: cross Abstract: Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state...
arXiv:2607. 23556v1 Announce Type: cross Abstract: Temporal graphs are increasingly used to model dynamic systems in diverse domains such as social networks, financial networks, and traffic networks.
The paper investigates how information-theoretic and geometric compression unfolds over time in recurrent neural networks, specifically discretized LSTMs. It finds that compression is strongest at the final timesteps of many datasets and attributes this to three mechanisms: class separability, temporal stationarity of shared dynamics, and limited representational capacity. Through synthetic and real-world experiments, the authors demonstrate that these factors jointly determine the extent of temporal compression and its role as a regularizer when excess capacity exists, but can harm generalization when it consumes task-relevant memory.
arXiv:2607. 18620v1 Announce Type: cross Abstract: This paper develops temporal-causal unity (TCU), a framework connecting a process-philosophical thesis -- time is the ordered unfolding of causal change -- to an operational model of cognitive and social dynamics.
arXiv:2607. 09232v1 Announce Type: new Abstract: Temporal knowledge graphs (TKGs) represent evolving relational systems, whose underlying data-generating processes often change over time.
The paper introduces a formal definition of the structural role of latent factors in dynamical systems, describing where a latent factor is expressed, whether it receives influence from past observations, and whether it transmits influence to future observations. It addresses the challenge of correspondence across different structural identifications and proves that, under certain conditions, all three roles can be jointly identified using a common permutation at each time point. Building on this theory, the authors present RoleTrace, a framework that traces these structural roles in a shared latent coordinate system, and validate its effectiveness through synthetic experiments and real-world multiparty meeting data.
The paper introduces a control foundation model that treats control-experiment data as dynamic graphs, allowing both response prediction and control-input generation to be handled as a unified masked prediction problem. It defines a data space of dynamic graphs with connections, signal values, and missing values, and uses the Block Diagram Transformer (BDT) to combine temporal attention with graph neural networks. Trained on mixed experimental settings, the BDT successfully stabilized an unstable system via feedback control, outperforming fine‑tuned time‑series foundation models.
The paper introduces causal‑fate dynamics, a framework describing how influence generated in a system can be realized, remain latent, or be transformed by later dynamics. It presents an exact finite‑transport representation for these processes and applies the concept to three domains: a connectome‑constrained model of *Caenorhabditis elegans* suggesting unresolved neuronal influence may persist, an analysis of Internet routing showing that cross‑observer history retains predictive power beyond local state, and a Transformer architecture that transports and selectively realizes latent contextual influence while preserving language‑modeling performance. These studies collectively illustrate how unrealized influence can be modeled, observed, and harnessed in computational systems.
arXiv:2606. 18746v1 Announce Type: new Abstract: This paper develops a formal account of what generalist agents must store in memory in order to act near-optimally across multiple environments and goals.
arXiv:2601. 21135v2 Announce Type: replace Abstract: Temporal causal representation learning methods assume that causal mechanisms switch instantaneously between discrete domains, yet real-world systems often exhibit continuous mechanism transitions.
arXiv:2610.09509v1 Announce Type: cross Abstract: Latent actions supervise robot policies with action-like codes inferred from visual transitions, and their usefulness hinges on two questions: what a...
arXiv:2606. 05194v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being deployed to make decisions that require trading off near-term gains against long-term consequences, yet little is known about how they internally represent or resolve these tradeoffs.