Single-Stage Open-world Instance Segmentation with Cross-task Consistency Regularization

by   Xizhe Xue, et al.

Open-world instance segmentation (OWIS) aims to segment class-agnostic instances from images, which has a wide range of real-world applications such as autonomous driving. Most existing approaches follow a two-stage pipeline: performing class-agnostic detection first and then class-specific mask segmentation. In contrast, this paper proposes a single-stage framework to produce a mask for each instance directly. Also, instance mask annotations could be noisy in the existing datasets; to overcome this issue, we introduce a new regularization loss. Specifically, we first train an extra branch to perform an auxiliary task of predicting foreground regions (i.e. regions belonging to any object instance), and then encourage the prediction from the auxiliary branch to be consistent with the predictions of the instance masks. The key insight is that such a cross-task consistency loss could act as an error-correcting mechanism to combat the errors in annotations. Further, we discover that the proposed cross-task consistency loss can be applied to images without any annotation, lending itself to a semi-supervised learning method. Through extensive experiments, we demonstrate that the proposed method can achieve impressive results in both fully-supervised and semi-supervised settings. Compared to SOTA methods, the proposed method significantly improves the AP_100 score by 4.75% in UVO→UVO setting and 4.05% in COCO→UVO setting. In the case of semi-supervised learning, our model learned with only 30% labeled data, even outperforms its fully-supervised counterpart with 50% labeled data. The code will be released soon.


page 2

page 4

page 8

page 11

page 12

page 13


The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation

In this paper, we introduce a novel learning scheme named weakly semi-su...

Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining

Point cloud instance segmentation has achieved huge progress with the em...

Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation

Partially supervised instance segmentation aims to perform learning on l...

Cut-Paste Consistency Learning for Semi-Supervised Lesion Segmentation

Semi-supervised learning has the potential to improve the data-efficienc...

Mask-guided sample selection for Semi-Supervised Instance Segmentation

Image segmentation methods are usually trained with pixel-level annotati...

Polite Teacher: Semi-Supervised Instance Segmentation with Mutual Learning and Pseudo-Label Thresholding

We present Polite Teacher, a simple yet effective method for the task of...

Monocular 3D Detection with Geometric Constraints Embedding and Semi-supervised Training

In this work, we propose a novel single-shot and keypoints-based framewo...

Please sign up or login with your details

Forgot password? Click here to reset