A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection

by   Xiaoqi Zhao, et al.

Existing RGB-D salient object detection (SOD) approaches concentrate on the cross-modal fusion between the RGB stream and the depth stream. They do not deeply explore the effect of the depth map itself. In this work, we design a single stream network to directly use the depth map to guide early fusion and middle fusion between RGB and depth, which saves the feature encoder of the depth stream and achieves a lightweight and real-time model. We tactfully utilize depth information from two perspectives: (1) Overcoming the incompatibility problem caused by the great difference between modalities, we build a single stream encoder to achieve the early fusion, which can take full advantage of ImageNet pre-trained backbone model to extract rich and discriminative features. (2) We design a novel depth-enhanced dual attention module (DEDA) to efficiently provide the fore-/back-ground branches with the spatially filtered features, which enables the decoder to optimally perform the middle fusion. Besides, we put forward a pyramidally attended feature extraction module (PAFE) to accurately localize the objects of different scales. Extensive experiments demonstrate that the proposed model performs favorably against most state-of-the-art methods under different evaluation metrics. Furthermore, this model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a 384 × 384 image.


page 2

page 12

page 13

page 14


Middle-level Fusion for Lightweight RGB-D Salient Object Detection

Most existing RGB-D salient object detection (SOD) models require large ...

RGB-D Salient Object Detection via 3D Convolutional Neural Networks

RGB-D salient object detection (SOD) recently has attracted increasing r...

TANet: Transformer-based Asymmetric Network for RGB-D Salient Object Detection

Existing RGB-D SOD methods mainly rely on a symmetric two-stream CNN-bas...

Suppress and Balance: A Simple Gated Network for Salient Object Detection

Most salient object detection approaches use U-Net or feature pyramid ne...

WCCNet: Wavelet-integrated CNN with Crossmodal Rearranging Fusion for Fast Multispectral Pedestrian Detection

Multispectral pedestrian detection achieves better visibility in challen...

Self-Supervised Representation Learning for RGB-D Salient Object Detection

Existing CNNs-Based RGB-D Salient Object Detection (SOD) networks are al...

ACFNet: Adaptively-Cooperative Fusion Network for RGB-D Salient Object Detection

The reasonable employment of RGB and depth data show great significance ...

Please sign up or login with your details

Forgot password? Click here to reset