Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction

by   Sara Hatami Gazani, et al.

UAV-based intelligent data acquisition for 3D reconstruction and monitoring of infrastructure has been experiencing an increasing surge of interest due to the recent advancements in image processing and deep learning-based techniques. View planning is an essential part of this task that dictates the information capture strategy and heavily impacts the quality of the 3D model generated from the captured data. Recent methods have used prior knowledge or partial reconstruction of the target to accomplish view planning for active reconstruction; the former approach poses a challenge for complex or newly identified targets while the latter is computationally expensive. In this work, we present Bag-of-Views (BoV), a fully appearance-based model used to assign utility to the captured views for both offline dataset refinement and online next-best-view (NBV) planning applications targeting the task of 3D reconstruction. With this contribution, we also developed the View Planning Toolbox (VPT), a lightweight package for training and testing machine learning-based view planning frameworks, custom view dataset generation of arbitrary 3D scenes, and 3D reconstruction. Through experiments which pair a BoV-based reinforcement learning model with VPT, we demonstrate the efficacy of our model in reducing the number of required views for high-quality reconstructions in dataset refinement and NBV planning.


page 1

page 5

page 6

page 7


Active Object Reconstruction Using a Guided View Planner

Inspired by the recent advance of image-based object reconstruction usin...

3D-NVS: A 3D Supervision Approach for Next View Selection

We present a classification based approach for the next best view select...

Adaptive View Planning for Aerial 3D Reconstruction of Complex Scenes

With the proliferation of small aerial vehicles, acquiring close up aeri...

Onboard View Planning of a Flying Camera for High Fidelity 3D Reconstruction of a Moving Actor

Capturing and reconstructing a human actor's motion is important for fil...

Higher Order Function Networks for View Planning and Multi-View Reconstruction

We consider the problem of planning views for a robot to acquire images ...

StyleTRF: Stylizing Tensorial Radiance Fields

Stylized view generation of scenes captured casually using a camera has ...

The Surface Edge Explorer (SEE): A measurement-direct approach to next best view planning

High-quality observations of the real world are crucial for a variety of...

Please sign up or login with your details

Forgot password? Click here to reset