DeepAI AI Chat
Log In Sign Up

Spatiotemporal Deformable Models for Long-Term Complex Activity Detection

by   Salman Khan, et al.
Oxford Brookes University

Long-term complex activity recognition and localisation can be crucial for the decision-making process of several autonomous systems, such as smart cars and surgical robots. Nonetheless, most current methods are designed to merely localise short-term action/activities or combinations of atomic actions that only last for a few frames or seconds. In this paper, we address the problem of long-term complex activity detection via a novel deformable, spatiotemporal parts-based model. Our framework consists of three main building blocks: (i) action tube detection, (ii) the modelling of the deformable geometry of parts, and (iii) a sparsity mechanism. Firstly, action tubes are detected in a series of snippets using an action tube detector. Next, a new 3D deformable RoI pooling layer is designed for learning the flexible, deformable geometry of the constellation of parts. Finally, a sparsity strategy differentiates between activated and deactivate features. We also provide temporal complex activity annotation for the recently released ROAD autonomous driving dataset and the SARAS-ESAD surgical action dataset, to validate our method and show the adaptability of our framework to different domains. As they both contain long videos portraying long-term activities they can be used as benchmarks for future work in this area.


page 2

page 3

page 8


A Logic Programming Approach to Activity Recognition

We have been developing a system for recognising human activity given a ...

Action Detection from a Robot-Car Perspective

We present the new Road Event and Activity Detection (READ) dataset, des...

Compositional Structure Learning for Action Understanding

The focus of the action understanding literature has predominately been ...

Deformable Tube Network for Action Detection in Videos

We address the problem of spatio-temporal action detection in videos. Ex...

A Probabilistic Logic Programming Event Calculus

We present a system for recognising human activity given a symbolic repr...

Towards Holistic Surgical Scene Understanding

Most benchmarks for studying surgical interventions focus on a specific ...

Egocentric Object Manipulation Graphs

We introduce Egocentric Object Manipulation Graphs (Ego-OMG) - a novel r...