Knowledge Soft Integration for Multimodal Recommendation

by   Kai Ouyang, et al.

One of the main challenges in modern recommendation systems is how to effectively utilize multimodal content to achieve more personalized recommendations. Despite various proposed solutions, most of them overlook the mismatch between the knowledge gained from independent feature extraction processes and downstream recommendation tasks. Specifically, multimodal feature extraction processes do not incorporate prior knowledge relevant to recommendation tasks, while recommendation tasks often directly use these multimodal features as side information. This mismatch can lead to model fitting biases and performance degradation, which this paper refers to as the curse of knowledge problem. To address this issue, we propose using knowledge soft integration to balance the utilization of multimodal features and the curse of knowledge problem it brings about. To achieve this, we put forward a Knowledge Soft Integration framework for the multimodal recommendation, abbreviated as KSI, which is composed of the Structure Efficiently Injection (SEI) module and the Semantic Soft Integration (SSI) module. In the SEI module, we model the modality correlation between items using Refined Graph Neural Network (RGNN), and introduce a regularization term to reduce the redundancy of user/item representations. In the SSI module, we design a self-supervised retrieval task to further indirectly integrate the semantic knowledge of multimodal features, and enhance the semantic discrimination of item representations. Extensive experiments on three benchmark datasets demonstrate the superiority of KSI and validate the effectiveness of its two modules.


page 3

page 8


MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation

In most E-commerce platforms, whether the displayed items trigger the us...

Attention-guided Multi-step Fusion: A Hierarchical Fusion Network for Multimodal Recommendation

The main idea of multimodal recommendation is the rational utilization o...

A Pre-training Strategy for Recommendation

The side information of items has been shown to be effective in building...

Interest-Related Item Similarity Model Based on Multimodal Data for Top-N Recommendation

Nowadays, the recommendation systems are applied in the fields of e-comm...

An Efficient Approach to Informative Feature Extraction from Multimodal Data

One primary focus in multimodal feature extraction is to find the repres...

Semantic-Guided Feature Distillation for Multimodal Recommendation

Multimodal recommendation exploits the rich multimodal information assoc...

Ducho: A Unified Framework for the Extraction of Multimodal Features in Recommendation

In multimodal-aware recommendation, the extraction of meaningful multimo...

Please sign up or login with your details

Forgot password? Click here to reset