Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning

by   Xun Wang, et al.

A family of loss functions built on pair-based computation have been proposed in the literature which provide a myriad of solutions for deep metric learning. In this paper, we provide a general weighting framework for understanding recent pair-based loss functions. Our contributions are three-fold: (1) we establish a General Pair Weighting (GPW) framework, which casts the sampling problem of deep metric learning into a unified view of pair weighting through gradient analysis, providing a powerful tool for understanding recent pair-based loss functions; (2) we show that with GPW, various existing pair-based methods can be compared and discussed comprehensively, with clear differences and key limitations identified; (3) we propose a new loss called multi-similarity loss (MS loss) under the GPW, which is implemented in two iterative steps (i.e., mining and weighting). This allows it to fully consider three similarities for pair weighting, providing a more principled approach for collecting and weighting informative pairs. Finally, the proposed MS loss obtains new state-of-the-art performance on four image retrieval benchmarks, where it outperforms the most recent approaches, such as ABEKim_2018_ECCV and HTL by a large margin: 60.6 and 80.9 available at


page 1

page 2

page 3

page 4


The General Pair-based Weighting Loss for Deep Metric Learning

Deep metric learning aims at learning the distance metric between pair o...

Mean Field Theory in Deep Metric Learning

In this paper, we explore the application of mean field theory, a techni...

SentPWNet: A Unified Sentence Pair Weighting Network for Task-specific Sentence Embedding

Pair-based metric learning has been widely adopted to learn sentence emb...

Rethinking Deep Contrastive Learning with Embedding Memory

Pair-wise loss functions have been extensively studied and shown to cont...

HyP^2 Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval

Image retrieval has become an increasingly appealing technique with broa...

Dissecting the impact of different loss functions with gradient surgery

Pair-wise loss is an approach to metric learning that learns a semantic ...

Circle Loss: A Unified Perspective of Pair Similarity Optimization

This paper provides a pair similarity optimization viewpoint on deep fea...

Please sign up or login with your details

Forgot password? Click here to reset