arXiv Machine Learning
Jul 30
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
arXiv:2607. 26369v1 Announce Type: new Abstract: Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models.
T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation
T‑RoPE introduces a time‑aware Rotary Position Embedding for sequential recommendation, replacing index‑only rotations with timestamp‑based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non‑stationary key rotation. The authors prove that standard RoPE is time‑translation invariant and cannot capture seasonal contexts, while T‑RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks and an industrial‑scale e‑commerce dataset, T‑RoPE outperforms baselines by up to 130 % in HR@10 and delivers significant online lift in conversion and order count.
Related stories
arXiv AI
Sep 2
From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs
The paper introduces ReST, a recommendation‑native Transformer scaling framework designed to handle noisy, irregular, and sparsely supervised user behavior sequences in production ranking. ReST employs a dual‑gated attention encoder with rotary positional and temporal embeddings, and a lightweight cross decoder that decouples heavy encoding from fast decoding, enabling efficient compute‑once, decode‑many‑times ranking. Experiments on industrial and public benchmarks show that ReST outperforms traditional Transformer blocks, achieving higher accuracy and consistent scaling across sequence length, depth, and width, and a one‑week online A/B test on a production advertising platform yielded a 1.31% AUC lift and an 11.93% increase in a core revenue metric within a 50 ms P99 latency budget.
Hugging Face Trending Papers
Aug 17
Decoupled Temporal Encoding for Generative Recommendation
Positional encoding is a fundamental component of Transformer-based generative recommendation models, where user histories are modeled as autoregressive item sequences. Most positional encoding methods are inherited from natural language processing and mainly represent discrete item order.
arXiv AI
Aug 18
Decoupled Temporal Encoding for Generative Recommendation
arXiv:2608. 16274v1 Announce Type: cross Abstract: Positional encoding is a fundamental component of Transformer-based generative recommendation models, where user histories are modeled as autoregressive item sequences.
arXiv AI
Sep 25
DeGRe: Dense-supervised Generative Reranking for Recommendation
DeGRe is a dense‑supervised generative reranking framework designed to improve multi‑stage recommender systems by addressing label bias and credit assignment issues. It uses an offline Lookahead Evaluator with beam search to generate dense supervision signals, which are distilled into a lightweight Online Generator that can perform efficient greedy decoding at inference time. Experiments show that DeGRe outperforms baselines on public benchmarks and industrial datasets, and it has been successfully deployed on Taobao Flash Shopping to enhance online recommendations.
arXiv AI
Sep 21
Dual-Interest Sequential Product Recommendation With Multi-Granular SSM
The paper introduces DSRec, a dual‑interest sequential recommendation model that separates item representations into long‑term and short‑term semantic contexts. Long‑term embeddings capture stable preferences through historical aggregation, while short‑term embeddings focus on local session intent modulated by inter‑click time intervals. Each branch is processed by a distinct State Space Model— a full‑sequence Mamba for long‑term modeling and a time‑modulated SSM for short‑term dynamics— and a residual cross‑fusion mechanism aligns the two granularities while preserving their independence. Experiments on public benchmarks show that DSRec outperforms state‑of‑the‑art methods.
arXiv AI
Sep 10
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
SequenceO1 is an end‑to‑end framework that enables ultra‑long (up to 100K interactions) sequence modeling for recommendation systems. It compresses raw user histories into a fixed‑size sketch using Sketch Attention and then models short‑term and long‑term interests with Target‑to‑History Cross Attention. The system incorporates low‑rank caching, batching, pipeline lift, and a FlashSA kernel to keep training and inference efficient, achieving consistent offline and online performance gains when deployed at full traffic on Douyin.
Hugging Face Trending Papers
Jul 5
LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation
Large language models (LLMs) have recently emerged as powerful backbones for recommender systems by reformulating recommendation as a token-level generation task. Despite their promise, we identify a pervasive yet underexplored issue: $\textit{Length Bias}$.
arXiv Computer Vision
1d ago
SOLAR: SVD-Optimized Lifelong Attention for Recommendation
arXiv:2603.02561v2 Announce Type: replace-cross Abstract: Attention mechanism remains the defining operator in Transformers since it provides expressive global credit assignment, yet its quadratic co...
arXiv AI
Jul 7
LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation
arXiv:2607. 04270v1 Announce Type: cross Abstract: Large language models (LLMs) have recently emerged as powerful backbones for recommender systems by reformulating recommendation as a token-level generation task.
arXiv AI
Jun 9
Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings
arXiv:2509. 10534v3 Announce Type: replace-cross Abstract: The attention mechanism in a Transformer architecture matches key to query based on both content -- the what -- and position in a sequence -- the where.
arXiv AI
Sep 1
TAAL: Mitigating Early Beam Pruning in Generative Recommendation via Temporal Autoregressive Alignment
arXiv:2608.29179v1 Announce Type: cross Abstract: Generative recommendation encodes items as hierarchical semantic identifiers (SIDs) and retrieves the next item through autoregressive decoding. Stan...