Video-Based Action Recognition Using Rate-Invariant Analysis of Covariance Trajectories

by   Zhengwu Zhang, et al.

Statistical classification of actions in videos is mostly performed by extracting relevant features, particularly covariance features, from image frames and studying time series associated with temporal evolutions of these features. A natural mathematical representation of activity videos is in form of parameterized trajectories on the covariance manifold, i.e. the set of symmetric, positive-definite matrices (SPDMs). The variable execution-rates of actions implies variable parameterizations of the resulting trajectories, and complicates their classification. Since action classes are invariant to execution rates, one requires rate-invariant metrics for comparing trajectories. A recent paper represented trajectories using their transported square-root vector fields (TSRVFs), defined by parallel translating scaled-velocity vectors of trajectories to a reference tangent space on the manifold. To avoid arbitrariness of selecting the reference and to reduce distortion introduced during this mapping, we develop a purely intrinsic approach where SPDM trajectories are represented by redefining their TSRVFs at the starting points of the trajectories, and analyzed as elements of a vector bundle on the manifold. Using a natural Riemannain metric on vector bundles of SPDMs, we compute geodesic paths and geodesic distances between trajectories in the quotient space of this vector bundle, with respect to the re-parameterization group. This makes the resulting comparison of trajectories invariant to their re-parameterization. We demonstrate this framework on two applications involving video classification: visual speech recognition or lip-reading and hand-gesture recognition. In both cases we achieve results either comparable to or better than the current literature.


page 3

page 13

page 16

page 19

page 20

page 22

page 24


Elastic Functional Coding of Riemannian Trajectories

Visual observations of dynamic phenomena, such as human actions, are oft...

Fitting, Comparison, and Alignment of Trajectories on Positive Semi-Definite Matrices with Application to Action Recognition

In this paper, we tackle the problem of action recognition using body sk...

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

In this paper, we propose a novel space-time geometric representation of...

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

In this paper, we propose a new approach for facial expression recogniti...

Online Action Recognition based on Incremental Learning of Weighted Covariance Descriptors

Different from traditional action recognition based on video segments, o...

Automatic Estimation of Self-Reported Pain by Trajectory Analysis in the Manifold of Fixed Rank Positive Semi-Definite Matrices

We propose an automatic method to estimate self-reported pain based on f...

An Elastic Energy Minimization Framework for Mean Contour Calculation

In this paper we propose a contour mean calculation and interpolation me...

Please sign up or login with your details

Forgot password? Click here to reset