PanoVOS:Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentation

by   Shilin Yan, et al.

Panoramic videos contain richer spatial information and have attracted tremendous amounts of attention due to their exceptional experience in some fields such as autonomous driving and virtual reality. However, existing datasets for video segmentation only focus on conventional planar images. To address the challenge, in this paper, we present a panoramic video dataset, PanoVOS. The dataset provides 150 videos with high video resolutions and diverse motions. To quantify the domain gap between 2D planar videos and panoramic videos, we evaluate 15 off-the-shelf video object segmentation (VOS) models on PanoVOS. Through error analysis, we found that all of them fail to tackle pixel-level content discontinues of panoramic videos. Thus, we present a Panoramic Space Consistency Transformer (PSCFormer), which can effectively utilize the semantic boundary information of the previous frame for pixel-level matching with the current frame. Extensive experiments demonstrate that compared with the previous SOTA models, our PSCFormer network exhibits a great advantage in terms of segmentation results under the panoramic setting. Our dataset poses new challenges in panoramic VOS and we hope that our PanoVOS can advance the development of panoramic segmentation/tracking.


page 3

page 7

page 8

page 10

page 12

page 13

page 14

page 15


Time-Space Transformers for Video Panoptic Segmentation

We propose a novel solution for the task of video panoptic segmentation,...

MSnet: Mutual Suppression Network for Disentangled Video Representations

The extraction of meaningful features from videos is important as they c...

Video Object of Interest Segmentation

In this work, we present a new computer vision task named video object o...

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations

We introduce VISOR, a new dataset of pixel annotations and a benchmark s...

Implicit Motion Handling for Video Camouflaged Object Detection

We propose a new video camouflaged object detection (VCOD) framework tha...

It is not Sexually Suggestive, It is Educative. Separating Sex Education from Suggestive Content on TikTok Videos

We introduce SexTok, a multi-modal dataset composed of TikTok videos lab...

Motion-state Alignment for Video Semantic Segmentation

In recent years, video semantic segmentation has made great progress wit...

Please sign up or login with your details

Forgot password? Click here to reset