DeepAI AI Chat
Log In Sign Up

Spatio-Temporal Action Localization in a Weakly Supervised Setting

by   Kurt Degiorgio, et al.
Oxford Brookes University

Enabling computational systems with the ability to localize actions in video-based content has manifold applications. Traditionally, such a problem is approached in a fully-supervised setting where video-clips with complete frame-by-frame annotations around the actions of interest are provided for training. However, the data requirements needed to achieve adequate generalization in this setting is prohibitive. In this work, we circumvent this issue by casting the problem in a weakly supervised setting, i.e., by considering videos as labelled `sets' of unlabelled video segments. Firstly, we apply unsupervised segmentation to take advantage of the elementary structure of each video. Subsequently, a convolutional neural network is used to extract RGB features from the resulting video segments. Finally, Multiple Instance Learning (MIL) is employed to predict labels at the video segment level, thus inherently performing spatio-temporal action detection. In contrast to previous work, we make use of a different MIL formulation in which the label of each video segment is continuous rather then discrete, making the resulting optimization function tractable. Additionally, we utilize a set splitting technique for regularization. Experimental results considering multiple performance indicators on the UCF-Sports data-set support the effectiveness of our approach.


page 3

page 6

page 7

page 9


Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos

Despite the recent advances in video classification, progress in spatio-...

Weakly Supervised Action Localization by Sparse Temporal Pooling Network

We propose a weakly supervised temporal action localization algorithm on...

Enabling Weakly-Supervised Temporal Action Localization from On-Device Learning of the Video Stream

Detecting actions in videos have been widely applied in on-device applic...

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Temporal action localization presents a trade-off between test performan...

Learning to Segment Actions from Observation and Narration

We apply a generative segmental model of task structure, guided by narra...

ReActNet: Temporal Localization of Repetitive Activities in Real-World Videos

We address the problem of temporal localization of repetitive activities...

Scalable Temporal Localization of Sensitive Activities in Movies and TV Episodes

To help customers make better-informed viewing choices, video-streaming ...