Machine-Generated Hierarchical Structure of Human Activities to Reveal How Machines Think

by   Mahsun Altın, et al.

Deep-learning based computer vision models have proved themselves to be ground-breaking approaches to human activity recognition (HAR). However, most existing works are dedicated to improve the prediction accuracy through either creating new model architectures, increasing model complexity, or refining model parameters by training on larger datasets. Here, we propose an alternative idea, differing from existing work, to increase model accuracy and also to shape model predictions to align with human understandings through automatically creating higher-level summarizing labels for similar groups of human activities. First, we argue the importance and feasibility of constructing a hierarchical labeling system for human activity recognition. Then, we utilize the predictions of a black box HAR model to identify similarities between different activities. Finally, we tailor hierarchical clustering methods to automatically generate hierarchical trees of activities and conduct experiments. In this system, the activity labels on the same level will have a designed magnitude of accuracy and reflect a specific amount of activity details. This strategy enables a trade-off between the extent of the details in the recognized activity and the user privacy by masking some sensitive predictions; and also provides possibilities for the use of formerly prohibited invasive models in privacy-concerned scenarios. Since the hierarchy is generated from the machine's perspective, the predictions at the upper levels provide better accuracy, which is especially useful when there are too detailed labels in the training set that are rather trivial to the final prediction goal. Moreover, the analysis of the structure of these trees can reveal the biases in the prediction model and guide future data collection strategies.


page 2

page 4

page 5

page 6

page 7

page 8

page 9

page 10


CHARM: A Hierarchical Deep Learning Model for Classification of Complex Human Activities Using Motion Sensors

In this paper, we report a hierarchical deep learning model for classifi...

Group Activity Recognition in Computer Vision: A Comprehensive Review, Challenges, and Future Perspectives

Group activity recognition is a hot topic in computer vision. Recognizin...

Cheating off your neighbors: Improving activity recognition through corroboration

Understanding the complexity of human activities solely through an indiv...

AudioAR: Audio-Based Activity Recognition with Large-Scale Acoustic Embeddings from YouTube Videos

Activity sensing and recognition have been demonstrated to be critical i...

Latent Hierarchical Model for Activity Recognition

We present a novel hierarchical model for human activity recognition. In...

Accoustate: Auto-annotation of IMU-generated Activity Signatures under Smart Infrastructure

Human activities within smart infrastructures generate a vast amount of ...

Please sign up or login with your details

Forgot password? Click here to reset