arXiv AI August 17, 2026 By Keito Kozaki, Keigo Sakurai, Ren Togo, Takahiro Ogawa, Miki Haseyama

Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders

arXiv:2608. 14021v1 Announce Type: new Abstract: Transformer-based sequential recommenders with causal self-attention often rely heavily on the most recent interaction at inference time, but how this behavior is structurally expressed in the representation used for prediction remains unclear.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

llms

arXiv Machine Learning

Jul 28

SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation

arXiv:2607. 24025v1 Announce Type: cross Abstract: Transformer architectures have achieved remarkable success across diverse domains; however, directly applying their standard self-attention mechanism to recommendation often yields suboptimal performance, sometimes even trailing behind well-designed simple recommendation models.

By Yu Cui, Yi Xu, Jiahao Wang, Hao Zhang, Yu Zhang, Xiaoyi Zeng, Can Wang, Jinxin Hu, Jiawei Chen

llms rag benchmarks safety

arXiv Machine Learning

Aug 5

Target-Aware Early Stage Ranking

arXiv:2511. 21095v2 Announce Type: replace Abstract: Early Stage Ranking (ESR) in large-scale recommendation systems is dominated by ''user--item decoupling'' Two Tower architectures, which scale efficiently but cannot capture fine-grained, target-aware user--item interactions directly.

By Juhee Hong, Meng Liu, Shengzhi Wang, Jin Zhou, Xiaoheng Mao, Zhao Zhu, Ruochen Liu, Huihui Cheng, Leon Gao, Christopher Leung, Chandra Mouli Sekar, Yijia Liu, Boyang Yu, Tuan Trieu, Dawei Sun, Jeet Kanjani, Rui Li, Jing Qian, Xuan Cao, Minjie Fan, Mingze Gao

efficiency

arXiv AI

Jun 9

Contribution Weights: A Geometrical Analysis of Self-Attention Transformers

arXiv:2606. 07604v1 Announce Type: cross Abstract: Analyzing attention weights has become a standard approach for interpreting the information flow of Large Language Models (LLMs).

By Harry Jake Cunningham, Nicola Muca Cirone

llms safety

arXiv Machine Learning

Jul 30

Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training

arXiv:2505. 15548v2 Announce Type: replace Abstract: Autoregressive transformer language models frequently exhibit training instability when trained on long sequences, particularly under low-precision arithmetic.

By Suvadeep Hajra

llms

arXiv Machine Learning

Jun 4

Customizing the Inductive Biases of Softmax Attention using Structured Matrices

arXiv:2509. 07963v2 Announce Type: replace Abstract: The core component of attention is the scoring function, which transforms the inputs into low-dimensional queries and keys and takes the dot product of each pair.

By Yilun Kuang, Noah Amsel, Sanae Lotfi, Shikai Qiu, Andres Potapczynski, Andrew Gordon Wilson

safety

arXiv AI

Jul 3

Self-Gating Attention for Efficient Time Series Forecasting

arXiv:2607. 02344v1 Announce Type: cross Abstract: Transformer architectures have shown strong potential in time series forecasting, where multi-head self-attention is widely used to capture temporal dependencies across historical timestamps.

By Dezheng Wang, Tong Chen, Wei Yuan, Congyan Chen, Shihua Li, Hongzhi Yin

llms efficiency benchmarks

Related stories

SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation

Target-Aware Early Stage Ranking

Contribution Weights: A Geometrical Analysis of Self-Attention Transformers

Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training

Customizing the Inductive Biases of Softmax Attention using Structured Matrices

Self-Gating Attention for Efficient Time Series Forecasting