Depth Assisted Full Resolution Network for Single Image-based View Synthesis

by   Xiaodong Cun, et al.

Researches in novel viewpoint synthesis majorly focus on interpolation from multi-view input images. In this paper, we focus on a more challenging and ill-posed problem that is to synthesize novel viewpoints from one single input image. To achieve this goal, we propose a novel deep learning-based technique. We design a full resolution network that extracts local image features with the same resolution of the input, which contributes to derive high resolution and prevent blurry artifacts in the final synthesized images. We also involve a pre-trained depth estimation network into our system, and thus 3D information is able to be utilized to infer the flow field between the input and the target image. Since the depth network is trained by depth order information between arbitrary pairs of points in the scene, global image features are also involved into our system. Finally, a synthesis layer is used to not only warp the observed pixels to the desired positions but also hallucinate the missing pixels with recorded pixels. Experiments show that our technique performs well on images of various scenes, and outperforms the state-of-the-art techniques.


page 2

page 3

page 5

page 6

page 7

page 8


Layer-structured 3D Scene Inference via View Synthesis

We present an approach to infer a layer-structured 3D representation of ...

Deep 3D Pan via adaptive "t-shaped" convolutions with global and local adaptive dilations

Recent advances in deep learning have shown promising results in many lo...

Enhancement of Novel View Synthesis Using Omnidirectional Image Completion

We present a method for synthesizing novel views from a single 360-degre...

Deep Depth from Focus with Differential Focus Volume

Depth-from-focus (DFF) is a technique that infers depth using the focus ...

DeepStereo: Learning to Predict New Views from the World's Imagery

Deep networks have recently enjoyed enormous success when applied to rec...

Deep Learning-based High-precision Depth Map Estimation from Missing Viewpoints for 360 Degree Digital Holography

In this paper, we propose a novel, convolutional neural network model to...

Please sign up or login with your details

Forgot password? Click here to reset