ConvFormer: Revisiting Transformer for Sequential User Modeling

by   Hao Wang, et al.

Sequential user modeling, a critical task in personalized recommender systems, focuses on predicting the next item a user would prefer, requiring a deep understanding of user behavior sequences. Despite the remarkable success of Transformer-based models across various domains, their full potential in comprehending user behavior remains untapped. In this paper, we re-examine Transformer-like architectures aiming to advance state-of-the-art performance. We start by revisiting the core building blocks of Transformer-based methods, analyzing the effectiveness of the item-to-item mechanism within the context of sequential user modeling. After conducting a thorough experimental analysis, we identify three essential criteria for devising efficient sequential user models, which we hope will serve as practical guidelines to inspire and shape future designs. Following this, we introduce ConvFormer, a simple but powerful modification to the Transformer architecture that meets these criteria, yielding state-of-the-art results. Additionally, we present an acceleration technique to minimize the complexity associated with processing extremely long sequences. Experiments on four public datasets showcase ConvFormer's superiority and confirm the validity of our proposed criteria.


page 1

page 2

page 3

page 4


Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Sequential Recommendation characterizes the evolving patterns by modelin...

Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation

Learning dynamic user preference has become an increasingly important co...

Denoising Self-attentive Sequential Recommendation

Transformer-based sequential recommenders are very powerful for capturin...

Seq2seq Translation Model for Sequential Recommendation

The context information such as product category plays a critical role i...

Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

The sequential patterns within the user interactions are pivotal for rep...

NxtPost: User to Post Recommendations in Facebook Groups

In this paper, we present NxtPost, a deployed user-to-post content-based...

ET-USB: Transformer-Based Sequential Behavior Modeling for Inbound Customer Service

Deep-Learning based models with attention mechanism has achieved excepti...

Please sign up or login with your details

Forgot password? Click here to reset