Improving Outfit Recommendation with Co-supervision of Fashion Generation

by   Yujie Lin, et al.

The task of fashion recommendation includes two main challenges: visual understanding and visual matching. Visual understanding aims to extract effective visual features. Visual matching aims to model a human notion of compatibility to compute a match between fashion items. Most previous studies rely on recommendation loss alone to guide visual understanding and matching. Although the features captured by these methods describe basic characteristics (e.g., color, texture, shape) of the input items, they are not directly related to the visual signals of the output items (to be recommended). This is problematic because the aesthetic characteristics (e.g., style, design), based on which we can directly infer the output items, are lacking. Features are learned under the recommendation loss alone, where the supervision signal is simply whether the given two items are matched or not. To address this problem, we propose a neural co-supervision learning framework, called the FAshion Recommendation Machine (FARM). FARM improves visual understanding by incorporating the supervision of generation loss, which we hypothesize to be able to better encode aesthetic information. FARM enhances visual matching by introducing a novel layer-to-layer matching mechanism to fuse aesthetic information more effectively, and meanwhile avoiding paying too much attention to the generation quality and ignoring the recommendation performance. Extensive experiments on two publicly available datasets show that FARM outperforms state-of-the-art models on outfit recommendation, in terms of AUC and MRR. Detailed analyses of generated and recommended items demonstrate that FARM can encode better features and generate high quality images as references to improve recommendation performance.


page 3

page 9


Explainable Fashion Recommendation with Joint Outfit Matching and Comment Generation

Most previous work on fashion recommendation focuses on designing visual...

Fashion Recommendation Based on Style and Social Events

Fashion recommendation is often declined as the task of finding compleme...

Visually-Aware Fashion Recommendation and Design with Generative Image Models

Building effective recommender systems for domains like fashion is chall...

Deep Set-to-Set Matching and Learning

Matching two sets of items, called set-to-set matching problem, is being...

Learning Tuple Compatibility for Conditional OutfitRecommendation

Outfit recommendation requires the answers of some challenging outfit co...

Addressing the Cold-Start Problem in Outfit Recommendation Using Visual Preference Modelling

With the global transformation of the fashion industry and a rise in the...

Transformer-based Graph Neural Networks for Outfit Generation

Suggesting complementary clothing items to compose an outfit is a proces...

Please sign up or login with your details

Forgot password? Click here to reset