Text Is All You Need: Learning Language Representations for Sequential Recommendation

by   Jiacheng Li, et al.

Sequential recommendation aims to model dynamic user behavior from historical interactions. Existing methods rely on either explicit item IDs or general textual features for sequence modeling to understand user preferences. While promising, these approaches still struggle to model cold-start items or transfer knowledge to new datasets. In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets. To this end, we present a novel framework, named Recformer, which effectively learns language representations for sequential recommendation. Specifically, we propose to formulate an item as a "sentence" (word sequence) by flattening item key-value attributes described by text so that an item sequence for a user becomes a sequence of sentences. For recommendation, Recformer is trained to understand the "sentence" sequence and retrieve the next "sentence". To encode item sequences, we design a bi-directional Transformer similar to the model Longformer but with different embedding layers for sequential recommendation. For effective representation learning, we propose novel pretraining and finetuning methods which combine language understanding and recommendation tasks. Therefore, Recformer can effectively recommend the next item based on language representations. Extensive experiments conducted on six datasets demonstrate the effectiveness of Recformer for sequential recommendation, especially in low-resource and cold-start settings.


page 1

page 2

page 3

page 4


Cold-start Sequential Recommendation via Meta Learner

This paper explores meta-learning in sequential recommendation to allevi...

Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning

Sequential recommendation systems utilize the sequential interactions of...

Towards Universal Sequence Representation Learning for Recommender Systems

In order to develop effective sequential recommenders, a series of seque...

Masked and Swapped Sequence Modeling for Next Novel Basket Recommendation in Grocery Shopping

Next basket recommendation (NBR) is the task of predicting the next set ...

Invariant representation learning for sequential recommendation

Sequential recommendation involves automatically recommending the next i...

Self-Supervised Multi-Modal Sequential Recommendation

With the increasing development of e-commerce and online services, perso...

Addressing the Rank Degeneration in Sequential Recommendation via Singular Spectrum Smoothing

Sequential recommendation (SR) investigates the dynamic user preferences...

Please sign up or login with your details

Forgot password? Click here to reset