Straight to Shapes++: Real-time Instance Segmentation Made More Accurate

by   Laurynas Miksys, et al.

Instance segmentation is an important problem in computer vision, with applications in autonomous driving, drone navigation and robotic manipulation. However, most existing methods are not real-time, complicating their deployment in time-sensitive contexts. In this work, we extend an existing approach to real-time instance segmentation, called `Straight to Shapes' (STS), which makes use of low-dimensional shape embedding spaces to directly regress to object shape masks. The STS model can run at 35 FPS on a high-end desktop, but its accuracy is significantly worse than that of offline state-of-the-art methods. We leverage recent advances in the design and training of deep instance segmentation models to improve the performance accuracy of the STS model whilst keeping its real-time capabilities intact. In particular, we find that parameter sharing, more aggressive data augmentation and the use of structured loss for shape mask prediction all provide a useful boost to the network performance. Our proposed approach, `Straight to Shapes++', achieves a remarkable 19.7 point improvement in mAP (at IOU of 0.5) over the original method as evaluated on the PASCAL VOC dataset, thus redefining the accuracy frontier at real-time speeds. Since the accuracy of instance segmentation is closely tied to that of object bounding box prediction, we also study the error profile of the latter and examine the failure modes of our method for future improvements.


page 7

page 10

page 20

page 21

page 22

page 23


Explicit Shape Encoding for Real-Time Instance Segmentation

In this paper, we propose a novel top-down instance segmentation framewo...

INSTA-YOLO: Real-Time Instance Segmentation

Instance segmentation has gained recently huge attention in various comp...

GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

We introduce a novel 3D object proposal approach named Generative Shape ...

Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation

Blastomere instance segmentation is important for analyzing embryos' abn...

Assessing YOLACT++ for real time and robust instance segmentation of medical instruments in endoscopic procedures

Image-based tracking of laparoscopic instruments plays a fundamental rol...

PanoNet: Real-time Panoptic Segmentation through Position-Sensitive Feature Embedding

We propose a simple, fast, and flexible framework to generate simultaneo...

Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN

Object recognition and instance segmentation are fundamental skills in a...

Please sign up or login with your details

Forgot password? Click here to reset