Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

by   Guanbin Li, et al.

Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.


page 1

page 4


Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

In facial action unit (AU) recognition tasks, regional feature learning ...

Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition

Humans can naturally understand an image in depth with the aid of rich k...

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

Facial Action Unit (AU) detection is a crucial task in affective computi...

Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition

The activations of Facial Action Units (AUs) mutually influence one anot...

Attention Based Relation Network for Facial Action Units Recognition

Facial action unit (AU) recognition is essential to facial expression an...

GeoConv: Geodesic Guided Convolution for Facial Action Unit Recognition

Automatic facial action unit (AU) recognition has attracted great attent...

Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition

Automatic facial action unit (AU) recognition is a challenging task due ...

Please sign up or login with your details

Forgot password? Click here to reset