arXiv AI

Time-Varying Data as Sheaves: an Invitation to Narratives

arXiv AI
Sep 10

Memory in Deep Time-Series Models

arXiv:2609.06006v1 Announce Type: cross Abstract: Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state...

By Minh Hoang Nguyen, Huu Hiep Nguyen, Manh Nguyen, Van Dai Do, Dung Nguyen, Hung Le
arXiv Machine Learning
Oct 22

Compression Through Time: An Interplay of Separability, Stationarity and Capacity

The paper investigates how information-theoretic and geometric compression unfolds over time in recurrent neural networks, specifically discretized LSTMs. It finds that compression is strongest at the final timesteps of many datasets and attributes this to three mechanisms: class separability, temporal stationarity of shared dynamics, and limited representational capacity. Through synthetic and real-world experiments, the authors demonstrate that these factors jointly determine the extent of temporal compression and its role as a regularizer when excess capacity exists, but can harm generalization when it consumes task-relevant memory.

By Nikolaus Kopp, Franz Pernkopf, Bernhard Geiger
arXiv Machine Learning
Oct 19

Where Latents Belong and How They Flow: Joint Identification of Structural Roles

The paper introduces a formal definition of the structural role of latent factors in dynamical systems, describing where a latent factor is expressed, whether it receives influence from past observations, and whether it transmits influence to future observations. It addresses the challenge of correspondence across different structural identifications and proves that, under certain conditions, all three roles can be jointly identified using a common permutation at each time point. Building on this theory, the authors present RoleTrace, a framework that traces these structural roles in a shared latent coordinate system, and validate its effectiveness through synthetic experiments and real-world multiparty meeting data.

By Xintong Li, Kun Zhang, Yujia Zheng
Hugging Face Trending Papers
Nov 2

Control Foundation Models across Changing Block Diagrams: Generalizing Time-Series Prediction to Dynamic Graphs

The paper introduces a control foundation model that treats control-experiment data as dynamic graphs, allowing both response prediction and control-input generation to be handled as a unified masked prediction problem. It defines a data space of dynamic graphs with connections, signal values, and missing values, and uses the Block Diagram Transformer (BDT) to combine temporal attention with graph neural networks. Trained on mixed experimental settings, the BDT successfully stabilized an unstable system via feedback control, outperforming fine‑tuned time‑series foundation models.

arXiv Machine Learning
Oct 9

Causal-fate dynamics of unrealized influence

The paper introduces causal‑fate dynamics, a framework describing how influence generated in a system can be realized, remain latent, or be transformed by later dynamics. It presents an exact finite‑transport representation for these processes and applies the concept to three domains: a connectome‑constrained model of *Caenorhabditis elegans* suggesting unresolved neuronal influence may persist, an analysis of Internet routing showing that cross‑observer history retains predictive power beyond local state, and a Transformer architecture that transports and selectively realizes latent contextual influence while preserving language‑modeling performance. These studies collectively illustrate how unrealized influence can be modeled, observed, and harnessed in computational systems.

By Yiwei Liu, Luwei Yang, Shunbo Lei
arXiv AI
Jun 18

What Must Generalist Agents Remember?

arXiv:2606. 18746v1 Announce Type: new Abstract: This paper develops a formal account of what generalist agents must store in memory in order to act near-optimally across multiple environments and goals.

By Khurram Yamin, Namrata Deka, Maitreyi Swaroop, Albert Ting, Jeff Schneider, Bryan Wilder
arXiv Machine Learning
Jun 5

Temporal Preference Concepts and their Functions in a Large Language Model

arXiv:2606. 05194v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being deployed to make decisions that require trading off near-term gains against long-term consequences, yet little is known about how they internally represent or resolve these tradeoffs.

By Ian Rios-Sialer, Shantanu Darveshi, Shuai Jiang, Avigya Paudel, Anastasiia Pronina, Ipshita Bandyopadhyay, Justin Shenk