arXiv AI
Screening Is Enough
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers
arXiv:2606. 29844v1 Announce Type: cross Abstract: The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particularly in long-context scenarios.
arXiv Computation and Language
Sep 14
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
arXiv:2609.13141v1 Announce Type: new Abstract: Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by selecting a small set of context u...
arXiv Machine Learning
Sep 29
Block Sparse Attention with Log-Linear Complexity
arXiv:2609. 31093v1 Announce Type: new Abstract: Scaling language models to long contexts is limited by the quadratic cost of self-attention.
arXiv Machine Learning
Oct 15
Continuum Attention
arXiv:2610.20586v1 Announce Type: new Abstract: We introduce Continuum Attention (CA), a causal Transformer attention operator that replaces discrete key and value lookup with integration over a cont...
Hugging Face Trending Papers
Aug 20
Learning how to Forget: Fine-tuning for Long-Context Sparse Attention
A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer language models without excessive hardware budgets. We provide a new method for fine-tuning models with sparse attention.
arXiv AI
Sep 16
QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge
QueryFormer is a unified architecture designed for post‑click conversion rate prediction, addressing both feature interactions and sequential user behaviors. It introduces a stackable field–sequence block that generates query tokens via cross‑attention and packs sequence queries into shared‑parameter attention, improving efficiency and accuracy. The model won first place in the KDD Cup 2026 Tencent UniRec Challenge Industrial Track with an AUC of 0.83254, and scaling studies show that increasing view width slightly boosts validation AUC while maintaining low latency.