Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation

by   Xiaoqi Zhao, et al.

Both static and moving objects usually exist in real-life videos. Most video object segmentation methods only focus on exacting and exploiting motion cues to perceive moving objects. Once faced with static objects frames, moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, many sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only utilize the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation. In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving both static and moving features used in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of optical flow and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor can precisely predicts a high-quality depth map and static saliency map at the same time.


page 2

page 3

page 5

page 11

page 12

page 13

page 15

page 16


Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation

Location and appearance are the key cues for video object segmentation. ...

Co-attention Propagation Network for Zero-Shot Video Object Segmentation

Zero-shot video object segmentation (ZS-VOS) aims to segment foreground ...

Dynamical optical flow of saliency maps for predicting visual attention

Saliency maps are used to understand human attention and visual fixation...

Thin-Plate Spline Motion Model for Image Animation

Image animation brings life to the static object in the source image acc...

Unsupervised Video Object Segmentation via Prototype Memory Network

Unsupervised video object segmentation aims to segment a target object i...

Reconstructing Articulated Rigged Models from RGB-D Videos

Although commercial and open-source software exist to reconstruct a stat...

Adversarial Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

We address the unsupervised learning of several interconnected problems ...

Please sign up or login with your details

Forgot password? Click here to reset