Addressing the Rank Degeneration in Sequential Recommendation via Singular Spectrum Smoothing

by   Ziwei Fan, et al.

Sequential recommendation (SR) investigates the dynamic user preferences modeling and generates the next-item prediction. The next item preference is typically generated by the affinity between the sequence and item representations. However, both sequence and item representations suffer from the rank degeneration issue due to the data sparsity problem. The rank degeneration issue significantly impairs the representations for SR. This motivates us to measure how severe is the rank degeneration issue and alleviate the sequence and item representation rank degeneration issues simultaneously for SR. In this work, we theoretically connect the sequence representation degeneration issue with the item rank degeneration, particularly for short sequences and cold items. We also identify the connection between the fast singular value decay phenomenon and the rank collapse issue in transformer sequence output and item embeddings. We propose the area under the singular value curve metric to evaluate the severity of the singular value decay phenomenon and use it as an indicator of rank degeneration. We further introduce a novel singular spectrum smoothing regularization to alleviate the rank degeneration on both sequence and item sides, which is the Singular sPectrum sMoothing for sequential Recommendation (SPMRec). We also establish a correlation between the ranks of sequence and item embeddings and the rank of the user-item preference prediction matrix, which can affect recommendation diversity. We conduct experiments on four benchmark datasets to demonstrate the superiority of SPMRec over the state-of-the-art recommendation methods, especially in short sequences. The experiments also demonstrate a strong connection between our proposed singular spectrum smoothing and recommendation diversity.


page 1

page 2

page 3

page 4


Text Is All You Need: Learning Language Representations for Sequential Recommendation

Sequential recommendation aims to model dynamic user behavior from histo...

Inter-sequence Enhanced Framework for Personalized Sequential Recommendation

Modeling the sequential correlation of users' historical interactions is...

Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation

Recent advancements of sequential deep learning models such as Transform...

Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation

Session-based recommendation (SR) has become an important and popular co...

DDGHM: Dual Dynamic Graph with Hybrid Metric Training for Cross-Domain Sequential Recommendation

Sequential Recommendation (SR) characterizes evolving patterns of user b...

Improving Transformer-based Sequential Recommenders through Preference Editing

One of the key challenges in Sequential Recommendation (SR) is how to ex...

Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation

Sequential recommendation is an important task to predict the next-item ...

Please sign up or login with your details

Forgot password? Click here to reset