NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding

by   Yongqi Zhang, et al.

Knowledge Graph (KG) embedding is a fundamental problem in data mining research with many real-world applications. It aims to encode the entities and relations in the graph into low dimensional vector space, which can be used for subsequent algorithms. Negative sampling, which samples negative triplets from non-observed ones in the training data, is an important step in KG embedding. Recently, generative adversarial network (GAN), has been introduced in negative sampling. By sampling negative triplets with large scores, these methods avoid the problem of vanishing gradient and thus obtain better performance. However, using GAN makes the original model more complex and hard to train, where reinforcement learning must be used. In this paper, motivated by the observation that negative triplets with large scores are important but rare, we propose to directly keep track of them with the cache. However, how to sample from and update the cache are two important questions. We carefully design the solutions, which are not only efficient but also achieve a good balance between exploration and exploitation. In this way, our method acts as a "distilled" version of previous GA-based methods, which does not waste training time on additional parameters to fit the full distribution of negative triplets. The extensive experiments show that our method can gain significant improvement in various KG embedding models, and outperform the state-of-the-art negative sampling methods based on GAN.


Efficient, Simple and Automated Negative Sampling for Knowledge Graph Embedding

Negative sampling, which samples negative triplets from non-observed one...

MixKG: Mixing for harder negative samples in knowledge graph

Knowledge graph embedding (KGE) aims to represent entities and relations...

Efficient Knowledge Graph Accuracy Evaluation

Estimation of the accuracy of a large-scale knowledge graph (KG) often r...

Defeats GAN: A Simpler Model Outperforms in Knowledge Representation Learning

The goal of knowledge representation learning is to embed entities and r...

Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

Knowledge graph embedding (KGE) has drawn great attention due to its pot...

GANE: A Generative Adversarial Network Embedding

Network embedding has become a hot research topic recently which can pro...

A Robust and Generalized Framework for Adversarial Graph Embedding

Graph embedding is essential for graph mining tasks. With the prevalence...

Please sign up or login with your details

Forgot password? Click here to reset