arXiv:2607. 12789v1 Announce Type: new Abstract: The $\mathcal{O}(N^2)$ complexity of attention over $N$ tokens remains a computational bottleneck in transformer models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
FlashLoop is a training‑free inference framework for Looped Transformers that reduces cross‑loop redundancy by employing token‑sparse updates, sparse attention, and KV‑residual quantization. It exploits observations that, as loops progress, state changes concentrate on a small token subset, attention differences are dominated by a sparse key subset, and KV residuals become amenable to low‑bit quantization. The method achieves lossless accuracy with up to 1.64× speedup and 6× KV‑cache memory reduction across several Looped Transformer models.
arXiv:2608. 04074v1 Announce Type: cross Abstract: Long-context LLM decoding reads the key-value (KV) cache at every step.
The paper introduces a quantization technique for tabular foundation models that focuses on converting queries, keys, and values to FP8 and employing explicit FP8 matrix multiplication to accelerate attention calculations. It emphasizes aligning quantization errors between training and test rows to avoid accuracy loss, and demonstrates up to 1.7× speedup over 16‑bit kernels with no significant accuracy degradation on TabPFN‑v3 and TabICLv2 across TabArena and BeyondArena.
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
The paper introduces RAM‑Net, a mechanism that unifies KV‑cache quantization and linear attention by using soft assignments over a discrete address space to update continuous slot states. It shows that these soft assignments extend hard quantized matching to a separable read‑write overlap, locally approximating full‑attention similarity while enabling recurrent aggregation. The authors demonstrate that RAM‑Net can migrate pretrained Transformer weights and recover an average of 87.1% of the original accuracy across nine models using only a 500M‑token budget.
arXiv:2608. 02691v1 Announce Type: cross Abstract: The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important.