arXiv:2610. 00009v1 Announce Type: new Abstract: Frequency-collapse attention [Zeris, 2026e] achieves large gains over standard dot-product attention by replacing the Q/K dot product with a bandpass-filtered inner product at a learned frequency.
By Athanasios Zeris
arXiv:2608. 04678v1 Announce Type: cross Abstract: Papers 1-2 of the Kathleen series showed that a byte-level, attention-free architecture built from a wavetable encoder and multi-scale reverberant state can match strong baselines on classification at ~450-700K parameters, without pretraining.
By George Fountzoulas
The paper introduces QK Product Steering, a data‑free, training‑free method that edits the query‑key product in vision‑language models to reduce object hallucination. By suppressing a few dominant singular modes in selected middle layers and mapping the edited product back to query weights, the approach lowers hallucination rates without affecting inference cost. Experiments on three GQA‑based VLMs show a 4.0% average reduction in CHAIR$_s$, with the effect localized to symmetric mutual‑attention channels.
By Karn Tiwari, Varnith Chordia, Prathosh A P
arXiv:2606. 18587v1 Announce Type: cross Abstract: Decoder-only Transformers compute attention over the KV cache of preceding tokens.
By Zhiyuan Wang, Xuan Luo, Sirui Zeng, Xifeng Yan
arXiv:2607. 19368v1 Announce Type: new Abstract: Long-prompt inference remains expensive because prefill attention scales quadratically with sequence length.
By Ali Mahdavi, Azaseh Zamanifar, Amirfarhad Farhadi, Omid Kashefi
arXiv:2606. 27748v1 Announce Type: cross Abstract: Transformer models rely on attention mechanism to capture long-range dependencies but suffer from quadratic complexity, limiting their scalability to long sequences.
By Haoran Zhang, Feng Zhou