Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition

Emotion recognition (ER) from facial images is one of the landmark tasks in affective computing with major developments in the last decade. Initial efforts on ER relied on handcrafted features that were used to characterize facial images and then feed to standard predictive models. Recent methodologies comprise end-to-end trainable deep learning methods that simultaneously learn both, features and predictive model. Perhaps the most successful models are based on convolutional neural networks (CNNs). While these models have excelled at this task, they still fail at capturing local patterns that could emerge in the learning process. We hypothesize these patterns could be captured by variants based on locally weighted learning. Specifically, in this paper we propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units that aims at exploiting local information at the final stage of the learning process. Intuitively, these RBF units capture local patterns shared by similar instances using an intermediate representation, then the outputs of the RBFs are feed to a softmax layer that exploits this information to improve the predictive performance of the model. This feature could be particularly advantageous in ER as cultural / ethnicity differences may be identified by the local units. We evaluate the proposed method in several ER datasets and show the proposed methodology achieves state-of-the-art in some of them, even when we adopt a pre-trained VGG-Face model as backbone. We show it is the incorporation of local information what makes the proposed model competitive.


page 7

page 8

page 9

page 10

page 11

page 18


Facial Emotion Recognition: A multi-task approach using deep learning

Facial Emotion Recognition is an inherently difficult problem, due to va...

Facial Emotion Recognition Using Deep Learning

We aim to construct a system that captures real-world facial images thro...

I Know How You Feel: Emotion Recognition with Facial Landmarks

Classification of human emotions remains an important and challenging ta...

Facial Emotion Recognition: State of the Art Performance on FER2013

Facial emotion recognition (FER) is significant for human-computer inter...

Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition

Automated emotion recognition in the wild from facial images remains a c...

Improving Facial Emotion Recognition Systems Using Gradient and Laplacian Images

In this work, we have proposed several enhancements to improve the perfo...

3DPalsyNet: A Facial Palsy Grading and Motion Recognition Framework using Fully 3D Convolutional Neural Networks

The capability to perform facial analysis from video sequences has signi...

Please sign up or login with your details

Forgot password? Click here to reset