Higher Order Function Networks for View Planning and Multi-View Reconstruction

by   Selim Engin, et al.

We consider the problem of planning views for a robot to acquire images of an object for visual inspection and reconstruction. In contrast to offline methods which require a 3D model of the object as input or online methods which rely on only local measurements, our method uses a neural network which encodes shape information for a large number of objects. We build on recent deep learning methods capable of generating a complete 3D reconstruction of an object from a single image. Specifically, in this work, we extend a recent method which uses Higher Order Functions (HOF) to represent the shape of the object. We present a new generalization of this method to incorporate multiple images as input and establish a connection between visibility and reconstruction quality. This relationship forms the foundation of our view planning method where we compute viewpoints to visually cover the output of the multi-view HOF network with as few images as possible. Experiments indicate that our method provides a good compromise between online and offline methods: Similar to online methods, our method does not require the true object model as input. In terms of number of views, it is much more efficient. In most cases, its performance is comparable to the optimal offline case even on object classes the network has not been trained on.


page 1

page 4

page 5


RealFusion: 360° Reconstruction of Any Object from a Single Image

We consider the problem of reconstructing a full 360 photographic model ...

CoRF : Colorizing Radiance Fields using Knowledge Distillation

Neural radiance field (NeRF) based methods enable high-quality novel-vie...

Higher-Order Function Networks for Learning Composable 3D Object Representations

We present a method to represent 3D objects using higher order functions...

Specular-to-Diffuse Translation for Multi-View Reconstruction

Most multi-view 3D reconstruction algorithms, especially when shape-from...

One-Shot View Planning for Fast and Complete Unknown Object Reconstruction

Current view planning (VP) systems usually adopt an iterative pipeline w...

Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction

UAV-based intelligent data acquisition for 3D reconstruction and monitor...

GARNet: Global-Aware Multi-View 3D Reconstruction Network and the Cost-Performance Tradeoff

Deep learning technology has made great progress in multi-view 3D recons...

Please sign up or login with your details

Forgot password? Click here to reset