RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State
Read the original on arXiv Machine Learning →RAM‑Net introduces a linear‑time sequence model that replaces dense access to a shared recurrent state with sparse, address‑based access. The model arranges the state as a fixed‑size array of independent slots and uses an Address Decoder to map each token’s key or query to a small subset of slots, thereby reducing inter‑token interference. Empirical results show that RAM‑Net outperforms strong recurrent baselines on fine‑grained long‑range retrieval, achieves the lowest perplexity in commonsense reasoning, and accesses far fewer state elements per step than competing models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.