Res-embedding for Deep Learning Based Click-Through Rate Prediction Modeling

by   Guorui Zhou, et al.

Recently, click-through rate (CTR) prediction models have evolved from shallow methods to deep neural networks. Most deep CTR models follow an Embedding&MLP paradigm, that is, first mapping discrete id features, e.g. user visited items, into low dimensional vectors with an embedding module, then learn a multi-layer perception (MLP) to fit the target. In this way, embedding module performs as the representative learning and plays a key role in the model performance. However, in many real-world applications, deep CTR model often suffers from poor generalization performance, which is mostly due to the learning of embedding parameters. In this paper, we model user behavior using an interest delay model, study carefully the embedding mechanism, and obtain two important results: (i) We theoretically prove that small aggregation radius of embedding vectors of items which belongs to a same user interest domain will result in good generalization performance of deep CTR model. (ii) Following our theoretical analysis, we design a new embedding structure named res-embedding. In res-embedding module, embedding vector of each item is the sum of two components: (i) a central embedding vector calculated from an item-based interest graph (ii) a residual embedding vector with its scale to be relatively small. Empirical evaluation on several public datasets demonstrates the effectiveness of the proposed res-embedding structure, which brings significant improvement on the model performance.


page 1

page 2

page 3

page 4


Item Cold Start Recommendation via Adversarial Variational Auto-encoder Warm-up

The gap between the randomly initialized item ID embedding and the well-...

Unstructured Semantic Model supported Deep Neural Network for Click-Through Rate Prediction

With the rapid development of online advertising and recommendation syst...

Feature embedding in click-through rate prediction

We tackle the challenge of feature embedding for the purposes of improvi...

Multi-Epoch Learning for Deep Click-Through Rate Prediction Models

The one-epoch overfitting phenomenon has been widely observed in industr...

Sparse-Interest Network for Sequential Recommendation

Recent methods in sequential recommendation focus on learning an overall...

DGEM: A New Dual-modal Graph Embedding Method in Recommendation System

In the current deep learning based recommendation system, the embedding ...

Fusion Strategies for Learning User Embeddings with Neural Networks

Growing amounts of online user data motivate the need for automated proc...

Please sign up or login with your details

Forgot password? Click here to reset