Sparse Attentive Memory Network for Click-through Rate Prediction with Long Sequences

by   Qianying Lin, et al.

Sequential recommendation predicts users' next behaviors with their historical interactions. Recommending with longer sequences improves recommendation accuracy and increases the degree of personalization. As sequences get longer, existing works have not yet addressed the following two main challenges. Firstly, modeling long-range intra-sequence dependency is difficult with increasing sequence lengths. Secondly, it requires efficient memory and computational speeds. In this paper, we propose a Sparse Attentive Memory (SAM) network for long sequential user behavior modeling. SAM supports efficient training and real-time inference for user behavior sequences with lengths on the scale of thousands. In SAM, we model the target item as the query and the long sequence as the knowledge database, where the former continuously elicits relevant information from the latter. SAM simultaneously models target-sequence dependencies and long-range intra-sequence dependencies with O(L) complexity and O(1) number of sequential updates, which can only be achieved by the self-attention mechanism with O(L^2) complexity. Extensive empirical results demonstrate that our proposed solution is effective not only in long user behavior modeling but also on short sequences modeling. Implemented on sequences of length 1000, SAM is successfully deployed on one of the largest international E-commerce platforms. This inference time is within 30ms, with a substantial 7.30 A/B test. To the best of our knowledge, it is the first end-to-end long user sequence modeling framework that models intra-sequence and target-sequence dependencies with the aforementioned degree of efficiency and successfully deployed on a large-scale real-time industrial recommender system.


page 1

page 2

page 3

page 4


Rethinking Lifelong Sequential Recommendation with Incremental Multi-Interest Attention

Sequential recommendation plays an increasingly important role in many e...

Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction

Recent studies on Click-Through Rate (CTR) prediction has reached new le...

Gating-adapted Wavelet Multiresolution Analysis for Exposure Sequence Modeling in CTR prediction

The exposure sequence is being actively studied for user interest modeli...

CmnRec: Sequential Recommendations with Chunk-accelerated Memory Network

Recently, Memory-based Neural Recommenders (MNR) have demonstrated super...

ChordMixer: A Scalable Neural Attention Model for Sequences with Different Lengths

Sequential data naturally have different lengths in many domains, with s...

Modeling the Field Value Variations and Field Interactions Simultaneously for Fraud Detection

With the explosive growth of e-commerce, online transaction fraud has be...

Time-based Sequence Model for Personalization and Recommendation Systems

In this paper we develop a novel recommendation model that explicitly in...

Please sign up or login with your details

Forgot password? Click here to reset