EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis

by   Gaëtan Landreau, et al.

Novel-view synthesis (NVS) can be tackled through different approaches, depending on the general setting: a single source image to a short video sequence, exact or noisy camera pose information, 3D-based information such as point clouds etc. The most challenging scenario, the one where we stand in this work, only considers a unique source image to generate a novel one from another viewpoint. However, in such a tricky situation, the latest learning-based solutions often struggle to integrate the camera viewpoint transformation. Indeed, the extrinsic information is often passed as-is, through a low-dimensional vector. It might even occur that such a camera pose, when parametrized as Euler angles, is quantized through a one-hot representation. This vanilla encoding choice prevents the learnt architecture from inferring novel views on a continuous basis (from a camera pose perspective). We claim it exists an elegant way to better encode relative camera pose, by leveraging 3D-related concepts such as the epipolar constraint. We, therefore, introduce an innovative method that encodes the viewpoint transformation as a 2D feature image. Such a camera encoding strategy gives meaningful insights to the network regarding how the camera has moved in space between the two views. By encoding the camera pose information as a finite number of coloured epipolar lines, we demonstrate through our experiments that our strategy outperforms vanilla encoding.


page 5

page 8

page 15

page 16

page 18

page 19

page 20

page 21


Novel View Synthesis from a Single Image via Unsupervised learning

View synthesis aims to generate novel views from one or more given sourc...

Consistent View Synthesis with Pose-Guided Diffusion Models

Novel view synthesis from a single image has been a cornerstone problem ...

Novel View Synthesis from only a 6-DoF Camera Pose by Two-stage Networks

Novel view synthesis is a challenging problem in computer vision and rob...

Euler angles based loss function for camera relocalization with Deep learning

Deep learning has been applied to camera relocalization, in particular, ...

Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints

Detecting 3D lanes from the camera is a rising problem for autonomous ve...

VMRF: View Matching Neural Radiance Fields

Neural Radiance Fields (NeRF) have demonstrated very impressive performa...

Gaussian Process Priors for View-Aware Inference

We derive a principled framework for encoding prior knowledge of informa...

Please sign up or login with your details

Forgot password? Click here to reset