arXiv:2607. 21927v1 Announce Type: new Abstract: Full self-attention in large language models scales as O(N^2), which limits long-context document analysis to 65,536 tokens and requires costly GPU clusters.
Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.
arXiv:2508. 18224v3 Announce Type: replace-cross Abstract: Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language models (LLMs).
arXiv:2606. 28876v1 Announce Type: cross Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory.
arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.
arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.
arXiv:2606. 25156v3 Announce Type: replace-cross Abstract: Native length extrapolation remain a weakly solvable problem in language modeling due to trade-off balancing between exact retrieval fidelity, long-document likelihood, and inference efficiency.
arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.