arXiv:2606. 18324v1 Announce Type: cross Abstract: SWave is a complex-valued recurrent language model (169.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones. In our diagnostic suite, the strongest efficient baselines tend to solve only one side well.
arXiv:2607. 21000v1 Announce Type: new Abstract: Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones.
arXiv:2608. 14691v1 Announce Type: new Abstract: Sequence models are conventionally distinguished by their backbone, the mechanism that routes information across positions, such as attention or recurrence.
arXiv:2607. 01394v1 Announce Type: new Abstract: We present Wiola, a fully original Small Language Model (SLM) architecture built from first principles, sharing no structural lineage with any existing model family including GPT, LLaMA, Mistral, or Falcon.
The paper introduces an input‑dependent readout phase for ResonatorLM, a resonant sequence model that traditionally uses a fixed phase per head to read memory. By making the phase a function of the current token, the authors add input dependence to the read‑only operation while preserving the exact convolutional path and adding only 0.2 % more parameters. Experiments on 6 M‑parameter character‑level models show that this modification matches the original performance at the training length and dramatically improves generalisation to longer sequences, reducing perplexity from 12.81 to 3.76 when trained on 256 tokens and tested on 1024 tokens.
arXiv:2609. 22415v1 Announce Type: new Abstract: Complex-valued Transformers have inherited softmax attention over the raw complex inner product.