RECipe: Does a Multi-Modal Recipe Knowledge Graph Fit a Multi-Purpose Recommendation System?

by   Ali Pesaranghader, et al.

Over the past two decades, recommendation systems (RSs) have used machine learning (ML) solutions to recommend items, e.g., movies, books, and restaurants, to clients of a business or an online platform. Recipe recommendation, however, has not yet received much attention compared to those applications. We introduce RECipe as a multi-purpose recipe recommendation framework with a multi-modal knowledge graph (MMKG) backbone. The motivation behind RECipe is to go beyond (deep) neural collaborative filtering (NCF) by recommending recipes to users when they query in natural language or by providing an image. RECipe consists of 3 subsystems: (1) behavior-based recommender, (2) review-based recommender, and (3) image-based recommender. Each subsystem relies on the embedding representations of entities and relations in the graph. We first obtain (pre-trained) embedding representations of textual entities, such as reviews or ingredients, from a fine-tuned model of Microsoft's MPNet. We initialize the weights of the entities with these embeddings to train our knowledge graph embedding (KGE) model. For the visual component, i.e., recipe images, we develop a KGE-Guided variational autoencoder (KG-VAE) to learn the distribution of images and their latent representations. Once KGE and KG-VAE models are fully trained, we use them as a multi-purpose recommendation framework. For benchmarking, we created two knowledge graphs (KGs) from public datasets on Kaggle for recipe recommendation. Our experiments show that the KGE models have comparable performance to the neural solutions. We also present pre-trained NLP embeddings to address important applications such as zero-shot inference for new users (or the cold start problem) and conditional recommendation with respect to recipe categories. We eventually demonstrate the application of RECipe in a multi-purpose recommendation setting.


page 14

page 19


Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation

Collaborative filtering often suffers from sparsity and cold start probl...

Bayes EMbedding (BEM): Refining Representation by Integrating Knowledge Graphs and Behavior-specific Networks

Low-dimensional embeddings of knowledge graphs and behavior graphs have ...

Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems

Modern neural collaborative filtering techniques are critical to the suc...

Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training

Existing recommender systems face difficulties with zero-shot items, i.e...

Representation Learning for Words and Entities

This thesis presents new methods for unsupervised learning of distribute...

Multi-modal Embedding Fusion-based Recommender

Recommendation systems have lately been popularized globally, with prima...

MultiHead MultiModal Deep Interest Recommendation Network

With the development of information technology, human beings are constan...

Please sign up or login with your details

Forgot password? Click here to reset