Temporal Event Segmentation using Attention-based Perceptual Prediction Model for Continual Learning

by   Ramy Mounir, et al.
University of South Florida

Temporal event segmentation of a long video into coherent events requires a high level understanding of activities' temporal features. The event segmentation problem has been tackled by researchers in an offline training scheme, either by providing full, or weak, supervision through manually annotated labels or by self-supervised epoch based training. In this work, we present a continual learning perceptual prediction framework (influenced by cognitive psychology) capable of temporal event segmentation through understanding of the underlying representation of objects within individual frames. Our framework also outputs attention maps which effectively localize and track events-causing objects in each frame. The model is tested on a wildlife monitoring dataset in a continual training manner resulting in 80% recall rate at 20% false positive rate for frame level segmentation. Activity level testing has yielded 80% activity recall rate for one false activity detection every 50 minutes.


page 1

page 3

page 8


Action Localization through Continual Predictive Learning

The problem of action recognition involves locating the action in the vi...

Polyphonic Sound Event and Sound Activity Detection: A Multi-task approach

Polyphonic Sound Event Detection (SED) in real-world recordings is a cha...

CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation

Some cognitive research has discovered that humans accomplish event segm...

Continual-wav2vec2: an Application of Continual Learning for Self-Supervised Automatic Speech Recognition

We present a method for continual learning of speech representations for...

Video Event Recognition for Surveillance Applications (VERSA)

VERSA provides a general-purpose framework for defining and recognizing ...

Automatic Detection Of Noise Events at Shooting Range Using Machine Learning

Outdoor shooting ranges are subject to noise regulations from local and ...

Please sign up or login with your details

Forgot password? Click here to reset