DanHAR: Dual Attention Network For Multimodal Human Activity Recognition Using Wearable Sensors

by   Wenbin Gao, et al.
Nanjing Normal University

Human activity recognition (HAR) in ubiquitous computing has been beginning to incorporate attention into the context of deep neural networks (DNNs), in which the rich sensing data from multimodal sensors such as accelerometer and gyroscope is used to infer human activities. Recently, two attention methods are proposed via combining with Gated Recurrent Units (GRU) and Long Short-Term Memory (LSTM) network, which can capture the dependencies of sensing signals in both spatial and temporal domains simultaneously. However, recurrent networks often have a weak feature representing power compared with convolutional neural networks (CNNs). On the other hand, two attention, i.e., hard attention and soft attention, are applied in temporal domains via combining with CNN, which pay more attention to the target activity from a long sequence. However, they can only tell where to focus and miss channel information, which plays an important role in deciding what to focus. As a result, they fail to address the spatial-temporal dependencies of multimodal sensing signals, compared with attention-based GRU or LSTM. In the paper, we propose a novel dual attention method called DanHAR, which introduces the framework of blending channel attention and temporal attention on a CNN, demonstrating superiority in improving the comprehensibility for multimodal HAR. Extensive experiments on four public HAR datasets and weakly labeled dataset show that DanHAR achieves state-of-the-art performance with negligible overhead of parameters. Furthermore, visualizing analysis is provided to show that our attention can amplifies more important sensor modalities and timesteps during classification, which agrees well with human common intuition.


page 1

page 7

page 9

page 10


An Attention-based Recurrent Convolutional Network for Vehicle Taillight Recognition

Vehicle taillight recognition is an important application for automated ...

Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention

Deep neural networks, including recurrent networks, have been successful...

Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework

Vision-based human activity recognition has emerged as one of the essent...

Sequential Weakly Labeled Multi-Activity Recognition and Location on Wearable Sensors using Recurrent Attention Network

With the popularity and development of the wearable devices such as smar...

Concurrent Activity Recognition with Multimodal CNN-LSTM Structure

We introduce a system that recognizes concurrent activities from real-wo...

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Multimodal sensors (e.g., visual, non-visual, and wearable) provide comp...

Encoding Accelerometer Signals as Images for Activity Recognition Using Residual Neural Network

Human activity recognition using a single 3-axis accelerometer plays an ...

Please sign up or login with your details

Forgot password? Click here to reset