Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition

by   Fanglei Xue, et al.

Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods.


page 1

page 3

page 6

page 8

page 11


NR-DFERNet: Noise-Robust Network for Dynamic Facial Expression Recognition

Dynamic facial expression recognition (DFER) in the wild is an extremely...

Learning Diversified Feature Representations for Facial Expression Recognition in the Wild

Diversity of the features extracted by deep neural networks is important...

Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator

Deriving an effective facial expression recognition component is importa...

Attentive Pooling Networks

In this work, we propose Attentive Pooling (AP), a two-way attention mec...

Quaternion Orthogonal Transformer for Facial Expression Recognition in the Wild

Facial expression recognition (FER) is a challenging topic in artificial...

Research on Patch Attentive Neural Process

Attentive Neural Process (ANP) improves the fitting ability of Neural Pr...

Please sign up or login with your details

Forgot password? Click here to reset