arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.
By Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua, Ning Ding, Xia Hu, Bowen Zhou, Chaochao Lu, Youbang Sun
SequenceO1 is an end‑to‑end framework that enables ultra‑long (up to 100K interactions) sequence modeling for recommendation systems. It compresses raw user histories into a fixed‑size sketch using Sketch Attention and then models short‑term and long‑term interests with Target‑to‑History Cross Attention. The system incorporates low‑rank caching, batching, pipeline lift, and a FlashSA kernel to keep training and inference efficient, achieving consistent offline and online performance gains when deployed at full traffic on Douyin.
By Lin Guan, Jia-Qi Yang, Zhishan Zhao, Jiaqi Huang, Hangyu Wang, Longbin Li, Beichuan Zhang, Haonan Jiang, Jinan Ni, Xiangyu Fan, Xiaowen Li, Ziyao Ren, Yuhang Qi, Xiaolong Zhu, Xuanyuan Luo, Qiwei Chen, Yi Cheng, Lele Yu
arXiv:2606. 07546v1 Announce Type: cross Abstract: Capturing user interests across extensive watch histories is critical for short-form video recommendation, yet scaling sequence length is limited by two bottlenecks: the semantic sparsity of atomic Video IDs and the quadratic computational complexity of Transformers.
By Ruixiao Sun, Diego Uribe Mora, Zhimeng Jiang, Yuanzhen Lin, Jiarui Wang, Yuening Li, Danfeng Guo, Zhizhong Chen, Chuan He, Liang Liu
BehaveRec is a three-signal framework that combines a causal session transformer, a bidirectional User History Encoder over a 90‑day window, and a GraphSAGE module built from behavioral embedding similarity. On an industrial dataset of 137M sequences, it improves R@5 by +3.2% over SASRec, and its advantage grows with user history length, becoming significant for users with 101+ interactions. The study also establishes a practical deployment rule of roughly 20 interactions per user for when long‑term history and behavioral‑graph signals are worthwhile.
By Aysenur Inan, Reza Yousefi Maragheh, Jayesh Uddhav Kudase, Selene Xu, Kaushiki Nag, Jianpeng Xu, Sushant Kumar
arXiv:2610.35323v1 Announce Type: cross
Abstract: Large language models incur substantial training and inference costs in personalized search and recommendation, as growing interaction histories leng...
By Haolin Li, Haipeng Zhang, Mang Li, Zhaohui Xia, Xun Zhu, Miao Li, Yu Zhang, Ji Wu
arXiv:2609.39007v1 Announce Type: cross
Abstract: Sessionized interaction histories contain behavioral patterns that can improve sequential recommendation. However, existing models process all sessio...
By Junyeong Song, Jaemin Yoo