arXiv Machine Learning
Nystr\"om Attention Matches Full Attention for Cross-Sectional Stock Prediction
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 5
Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and global attention models
arXiv:2508. 19006v2 Announce Type: replace-cross Abstract: This study investigates the pre-trained RNN attention models with the mainstream attention mechanisms, such as additive attention, Luong's three attentions, global self-attention and sliding window sparse attention, for the empirical asset pricing research on the top 420 large-cap US stocks.
arXiv Computation and Language
6d ago
Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction
arXiv:2609.30738v1 Announce Type: new Abstract: KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score,...
arXiv Machine Learning
Jun 24
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
arXiv:2504. 17768v3 Announce Type: replace-cross Abstract: Sparse attention offers a promising strategy to extend long-context capabilities in Transformer LLMs, yet its efficiency-accuracy trade-offs remain unclear due to the lack of comprehensive evaluation.
arXiv AI
Jul 22
A Controlled Study of Attention-Only Transformers
arXiv:2607. 18363v1 Announce Type: cross Abstract: Feed-forward networks hold two thirds of a transformer's non-embedding parameters, yet the architecture has not received a necessity test that controls parameters, compute, and depth at once.
arXiv AI
Jun 11
The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Content
arXiv:2606. 11198v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems inject external knowledge to improve LLM outputs, yet the format of injected content -- distinct from its semantic relevance -- can independently distort the model's attention distribution.
arXiv AI
Aug 11
Hybrid Neural-Classical Correction for Frozen Time Series Foundation Models: A Comprehensive Ablation Study on High-Frequency Stock Prediction
arXiv:2608. 08825v1 Announce Type: cross Abstract: Foundation models for time series forecasting demonstrate impressive zero-shot generalization but often underperform on specialized domains such as high-frequency finance.