arXiv:2607. 26369v1 Announce Type: new Abstract: Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
T‑RoPE introduces a time‑aware Rotary Position Embedding for sequential recommendation, replacing index‑only rotations with timestamp‑based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non‑stationary key rotation. The authors prove that standard RoPE is time‑translation invariant and cannot capture seasonal contexts, while T‑RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks and an industrial‑scale e‑commerce dataset, T‑RoPE outperforms baselines by up to 130 % in HR@10 and delivers significant online lift in conversion and order count.
Positional encoding is a fundamental component of Transformer-based generative recommendation models, where user histories are modeled as autoregressive item sequences. Most positional encoding methods are inherited from natural language processing and mainly represent discrete item order.
The paper introduces ReST, a recommendation‑native Transformer scaling framework designed to handle noisy, irregular, and sparsely supervised user behavior sequences in production ranking. ReST employs a dual‑gated attention encoder with rotary positional and temporal embeddings, and a lightweight cross decoder that decouples heavy encoding from fast decoding, enabling efficient compute‑once, decode‑many‑times ranking. Experiments on industrial and public benchmarks show that ReST outperforms traditional Transformer blocks, achieving higher accuracy and consistent scaling across sequence length, depth, and width, and a one‑week online A/B test on a production advertising platform yielded a 1.31% AUC lift and an 11.93% increase in a core revenue metric within a 50 ms P99 latency budget.
arXiv:2608. 16274v1 Announce Type: cross Abstract: Positional encoding is a fundamental component of Transformer-based generative recommendation models, where user histories are modeled as autoregressive item sequences.
The paper introduces DSRec, a dual‑interest sequential recommendation model that separates item representations into long‑term and short‑term semantic contexts. Long‑term embeddings capture stable preferences through historical aggregation, while short‑term embeddings focus on local session intent modulated by inter‑click time intervals. Each branch is processed by a distinct State Space Model— a full‑sequence Mamba for long‑term modeling and a time‑modulated SSM for short‑term dynamics— and a residual cross‑fusion mechanism aligns the two granularities while preserving their independence. Experiments on public benchmarks show that DSRec outperforms state‑of‑the‑art methods.
arXiv:2603.02561v2 Announce Type: replace-cross Abstract: Attention mechanism remains the defining operator in Transformers since it provides expressive global credit assignment, yet its quadratic co...