ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning

by   Zhiwu Qing, et al.

The central idea of contrastive learning is to discriminate between different instances and force different views of the same instance to share the same representation. To avoid trivial solutions, augmentation plays an important role in generating different views, among which random cropping is shown to be effective for the model to learn a strong and generalized representation. Commonly used random crop operation keeps the difference between two views statistically consistent along the training process. In this work, we challenge this convention by showing that adaptively controlling the disparity between two augmented views along the training process enhances the quality of the learnt representation. Specifically, we present a parametric cubic cropping operation, ParamCrop, for video contrastive learning, which automatically crops a 3D cubic from the video by differentiable 3D affine transformations. ParamCrop is trained simultaneously with the video backbone using an adversarial objective and learns an optimal cropping strategy from the data. The visualizations show that the center distance and the IoU between two augmented views are adaptively controlled by ParamCrop and the learned change in the disparity along the training process is beneficial to learning a strong representation. Extensive ablation studies demonstrate the effectiveness of the proposed ParamCrop on multiple contrastive learning frameworks and video backbones. With ParamCrop, we improve the state-of-the-art performance on both HMDB51 and UCF101 datasets.


page 3

page 13

page 14

page 15


Hierarchical Consistent Contrastive Learning for Skeleton-Based Action Recognition with Growing Augmentations

Contrastive learning has been proven beneficial for self-supervised skel...

Composable Augmentation Encoding for Video Representation Learning

We focus on contrastive methods for self-supervised video representation...

Meta-optimized Contrastive Learning for Sequential Recommendation

Contrastive Learning (CL) performances as a rising approach to address t...

Strongly Augmented Contrastive Clustering

Deep clustering has attracted increasing attention in recent years due t...

Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting

Instance-level contrastive learning techniques, which rely on data augme...

CCL4Rec: Contrast over Contrastive Learning for Micro-video Recommendation

Micro-video recommender systems suffer from the ubiquitous noises in use...

Adversarial Lagrangian Integrated Contrastive Embedding for Limited Size Datasets

Certain datasets contain a limited number of samples with highly various...

Please sign up or login with your details

Forgot password? Click here to reset