Learning a Metric Embedding for Face Recognition using the Multibatch Method

by   Oren Tadmor, et al.

This work is motivated by the engineering task of achieving a near state-of-the-art face recognition on a minimal computing budget running on an embedded system. Our main technical contribution centers around a novel training method, called Multibatch, for similarity learning, i.e., for the task of generating an invariant "face signature" through training pairs of "same" and "not-same" face images. The Multibatch method first generates signatures for a mini-batch of k face images and then constructs an unbiased estimate of the full gradient by relying on all k^2-k pairs from the mini-batch. We prove that the variance of the Multibatch estimator is bounded by O(1/k^2), under some mild conditions. In contrast, the standard gradient estimator that relies on random k/2 pairs has a variance of order 1/k. The smaller variance of the Multibatch estimator significantly speeds up the convergence rate of stochastic gradient descent. Using the Multibatch method we train a deep convolutional neural network that achieves an accuracy of 98.2% on the LFW benchmark, while its prediction runtime takes only 30msec on a single ARM Cortex A9 core. Furthermore, the entire training process took only 12 hours on a single Titan X GPU.


page 1

page 2

page 3

page 4


Accelerating Stochastic Gradient Descent Using Antithetic Sampling

(Mini-batch) Stochastic Gradient Descent is a popular optimization metho...

Neural Aggregation Network for Video Face Recognition

This paper presents a Neural Aggregation Network (NAN) for video face re...

A Study of Gradient Variance in Deep Learning

The impact of gradient noise on training deep models is widely acknowled...

Variance Suppression: Balanced Training Process in Deep Learning

Stochastic gradient descent updates parameters with summation gradient c...

Active Mini-Batch Sampling using Repulsive Point Processes

The convergence speed of stochastic gradient descent (SGD) can be improv...

Multi Modal Face Recognition Using Block Based Curvelet Features

In this paper, we present multimodal 2D +3D face recognition method usin...

Please sign up or login with your details

Forgot password? Click here to reset