3DPalsyNet: A Facial Palsy Grading and Motion Recognition Framework using Fully 3D Convolutional Neural Networks

by   Gary Storey, et al.

The capability to perform facial analysis from video sequences has significant potential to positively impact in many areas of life. One such area relates to the medical domain to specifically aid in the diagnosis and rehabilitation of patients with facial palsy. With this application in mind, this paper presents an end-to-end framework, named 3DPalsyNet, for the tasks of mouth motion recognition and facial palsy grading. 3DPalsyNet utilizes a 3D CNN architecture with a ResNet backbone for the prediction of these dynamic tasks. Leveraging transfer learning from a 3D CNNs pre-trained on the Kinetics data set for general action recognition, the model is modified to apply joint supervised learning using center and softmax loss concepts. 3DPalsyNet is evaluated on a test set consisting of individuals with varying ranges of facial palsy and mouth motions and the results have shown an attractive level of classification accuracy in these task of 82 duration and the loss function affect was studied in terms of the predictive qualities of the proposed 3DPalsyNet, where it was found shorter frame duration's of 8 performed best for this specific task. Centre loss and softmax have shown improvements in spatio-temporal feature learning than softmax loss alone, this is in agreement with earlier work involving the spatial domain.


page 2

page 3

page 6

page 8

page 9


Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild

Previous methods for dynamic facial expression in the wild are mainly ba...

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

This paper presents our Facial Action Units (AUs) recognition submission...

Transfer Learning for Action Unit Recognition

This paper presents a classifier ensemble for Facial Expression Recognit...

Theater Aid System for the Visually Impaired Through Transfer Learning of Spatio-Temporal Graph Convolution Networks

The aim of this research is to recognize human actions performed on stag...

Spatio-Temporal Crop Aggregation for Video Representation Learning

We propose Spatio-temporal Crop Aggregation for video representation LEa...

Relational Deep Feature Learning for Heterogeneous Face Recognition

Heterogeneous Face Recognition (HFR) is a task that matches faces across...

Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition

Emotion recognition (ER) from facial images is one of the landmark tasks...

Please sign up or login with your details

Forgot password? Click here to reset