arXiv Computation and Language By Youshan Zhang, Jialu Li

Time Delay System for Sequence Modeling

Read the original on arXiv Computation and Language →

The paper introduces a new sequence modeling framework based on nonlinear time-delay systems that dynamically estimates effective delay using semantic similarity between current and past inputs. It proposes a differentiable semantic lag estimator for content-aware soft attention, a diagonal state space with O(1) per-step complexity, and an interpretable fuzzy semantic modulation mechanism that balances immediate processing with historical recall. Experiments on text classification and generation show that this approach outperforms state‑of‑the‑art methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning
Aug 12

ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models

arXiv:2608. 10120v1 Announce Type: new Abstract: Modern sequence models, from Transformers to State Space Models, have enabled powerful generative modeling across diverse domains, yet they are typically trained to predict what happens while treating when it happens as a secondary concern.

By Adrien Schoen, Nachiketa Ratnakar Patil, Arjun Bhagoji, Francesco Bronzino
arXiv AI
Jun 10

Blurry Window Attention

arXiv:2606. 09862v1 Announce Type: cross Abstract: The Softmax Attention operation in Transformer language models has a quadratic complexity in the sequence length and a growing state size in the form of KV cache, which becomes a bottleneck in long context scenarios.

By Axel Laborieux, Christos Sourmpis, Juan Gabriel Kostelec, Qinghai Guo