Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image

by   Rui Zhu, et al.

An emerging problem in computer vision is the reconstruction of 3D shape and pose of an object from a single image. Hitherto, the problem has been addressed through the application of canonical deep learning methods to regress from the image directly to the 3D shape and pose labels. These approaches, however, are problematic from two perspectives. First, they are minimizing the error between 3D shapes and pose labels - with little thought about the nature of this label error when reprojecting the shape back onto the image. Second, they rely on the onerous and ill-posed task of hand labeling natural images with respect to 3D shape and pose. In this paper we define the new task of pose-aware shape reconstruction from a single image, and we advocate that cheaper 2D annotations of objects silhouettes in natural images can be utilized. We design architectures of pose-aware shape reconstruction which re-project the predicted shape back on to the image using the predicted pose. Our evaluation on several object categories demonstrates the superiority of our method for predicting pose-aware 3D shapes from natural images.


page 1

page 7


Self-supervised Learning of 3D Objects from Natural Images

We present a method to learn single-view reconstruction of the 3D shape,...

Optimizable Object Reconstruction from a Single View

3D shape reconstruction from a single image is a highly ill-posed proble...

Shape, Illumination, and Reflectance from Shading

A fundamental problem in computer vision is that of inferring the intrin...

Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion

Neural Radiance Fields (NeRF) coupled with GANs represent a promising di...

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

Natural images are projections of 3D objects on a 2D image plane. While ...

Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey

In nature, the collective behavior of animals, such as flying birds is d...

Automatic 3D Reconstruction for Symmetric Shapes

Generic 3D reconstruction from a single image is a difficult problem. A ...

Please sign up or login with your details

Forgot password? Click here to reset