AudioAR: Audio-Based Activity Recognition with Large-Scale Acoustic Embeddings from YouTube Videos

by   Dawei Liang, et al.

Activity sensing and recognition have been demonstrated to be critical in health care and smart home applications. Comparing to traditional methods such as using accelerometers or gyroscopes for activity recognition, acoustic-based methods can collect rich information of human activities together with the activity context, and therefore are more suitable for recognizing high-level compound activities. However, audio-based activity recognition in practice always suffers from the tedious and time-consuming process of collecting ground truth audio data from individual users. In this paper, we proposed a new mechanism of audio-based activity recognition that is entirely free from user training data by usage of millions of embedding features from general YouTube video sound clips. Based on combination of oversampling and deep learning approaches, our scheme does not require further feature extraction or outliers filtering for implementation. We developed our scheme for recognition of 15 common home-related activities and evaluated its performance under dedicated scenarios and in-the-wild scripted scenarios. In the dedicated recording test, our scheme yielded 81.1 activities. In the in-the-wild scripted tests, we obtained an averaged top-1 classification accuracy of 64.9 of 80.6 considerations including association between dataset labels and target activities, effects of segmentation size and privacy concerns were also discussed in the paper.


page 1

page 2

page 3

page 4


Audio-Based Activities of Daily Living (ADL) Recognition with Large-Scale Acoustic Embeddings from Online Videos

Over the years, activity sensing and recognition has been shown to play ...

Automated Activity Recognition of Construction Equipment Using a Data Fusion Approach

Automated monitoring of construction operations, especially operations o...

Accoustate: Auto-annotation of IMU-generated Activity Signatures under Smart Infrastructure

Human activities within smart infrastructures generate a vast amount of ...

Robust Activity Recognition for Adaptive Worker-Robot Interaction using Transfer Learning

Human activity recognition (HAR) using machine learning has shown tremen...

Machine-Generated Hierarchical Structure of Human Activities to Reveal How Machines Think

Deep-learning based computer vision models have proved themselves to be ...

Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight

State-of-the-art activity recognizers are effective during the day, but ...

The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary

The 3rd annual installment of the ActivityNet Large- Scale Activity Reco...

Please sign up or login with your details

Forgot password? Click here to reset