Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

by   Kai Wang, et al.
Dalian University of Technology
Macquarie University

Knowledge graph embedding (KGE) has drawn great attention due to its potential in automatic knowledge graph (KG) completion and knowledge-driven tasks. However, recent KGE models suffer from high training cost and large storage space, thus limiting their practicality in real-world applications. To address this challenge, based on the latest findings in the field of Contrastive Learning, we propose a novel KGE training framework called Hardness-aware Low-dimensional Embedding (HaLE). Instead of the traditional Negative Sampling, we design a new loss function based on query sampling that can balance two important training targets, Alignment and Uniformity. Furthermore, we analyze the hardness-aware ability of recent low-dimensional hyperbolic models and propose a lightweight hardness-aware activation mechanism, which can help the KGE models focus on hard instances and speed up convergence. The experimental results show that in the limited training time, HaLE can effectively improve the performance and training speed of KGE models on five commonly-used datasets. The HaLE-trained models can obtain a high prediction accuracy after training few minutes and are competitive compared to the state-of-the-art models in both low- and high-dimensional conditions.


page 1

page 2

page 3

page 4


Multi-teacher Knowledge Distillation for Knowledge Graph Completion

Link prediction based on knowledge graph embedding (KGE) aims to predict...

High-efficiency Euclidean-based Models for Low-dimensional Knowledge Graph Embeddings

Recent knowledge graph embedding (KGE) models based on hyperbolic geomet...

Investigating the Effect of Hard Negative Sample Distribution on Contrastive Knowledge Graph Embedding

The success of the knowledge graph completion task heavily depends on th...

Multiple Run Ensemble Learning withLow-Dimensional Knowledge Graph Embeddings

Among the top approaches of recent years, link prediction using knowledg...

ProjE: Embedding Projection for Knowledge Graph Completion

With the large volume of new information created every day, determining ...

NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding

Knowledge Graph (KG) embedding is a fundamental problem in data mining r...

Potential Destination Prediction Based on Knowledge Graph Under Low Predictability Data Condition

Destination prediction has been a critical topic in transportation resea...

Please sign up or login with your details

Forgot password? Click here to reset