Novel View Synthesis from a Single Image via Unsupervised learning

10/29/2021
by   Bingzheng Liu, et al.
Santa Clara University
University of Wollongong
Tianjin University
5

View synthesis aims to generate novel views from one or more given source views. Although existing methods have achieved promising performance, they usually require paired views of different poses to learn a pixel transformation. This paper proposes an unsupervised network to learn such a pixel transformation from a single source viewpoint. In particular, the network consists of a token transformation module (TTM) that facilities the transformation of the features extracted from a source viewpoint image into an intrinsic representation with respect to a pre-defined reference pose and a view generation module (VGM) that synthesizes an arbitrary view from the representation. The learned transformation allows us to synthesize a novel view from any single source viewpoint image of unknown pose. Experiments on the widely used view synthesis datasets have demonstrated that the proposed network is able to produce comparable results to the state-of-the-art methods despite the fact that learning is unsupervised and only a single source viewpoint image is required for generating a novel view. The code will be available soon.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

10/24/2022

EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis

Novel-view synthesis (NVS) can be tackled through different approaches, ...
09/29/2022

SymmNeRF: Learning to Explore Symmetry Prior for Single-View View Synthesis

We study the problem of novel view synthesis of objects from a single im...
04/15/2021

Geometry-Free View Synthesis: Transformers and no 3D Priors

Is a geometric model required to synthesize novel views from a single im...
07/13/2020

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

This paper targets on learning-based novel view synthesis from a single ...
11/22/2019

ViewSynth: Learning Local Features from Depth using View Synthesis

We address the problem of jointly detecting keypoints and learning descr...
06/07/2021

Novel View Video Prediction Using a Dual Representation

We address the problem of novel view video prediction; given a set of in...
04/11/2018

View Extrapolation of Human Body from a Single Image

We study how to synthesize novel views of human body from a single image...

Please sign up or login with your details

Forgot password? Click here to reset