Z2P: Instant Rendering of Point Clouds

05/30/2021
by   Gal Metzer, et al.
0

We present a technique for rendering point clouds using a neural network. Existing point rendering techniques either use splatting, or first reconstruct a surface mesh that can then be rendered. Both of these techniques require solving for global point normal orientation, which is a challenging problem on its own. Furthermore, splatting techniques result in holes and overlaps, whereas mesh reconstruction is particularly challenging, especially in the cases of thin surfaces and sheets. We cast the rendering problem as a conditional image-to-image translation problem. In our formulation, Z2P, i.e., depth-augmented point features as viewed from target camera view, are directly translated by a neural network to rendered images, conditioned on control variables (e.g., color, light). We avoid inevitable issues with splatting (i.e., holes and overlaps), and bypass solving the notoriously challenging surface reconstruction problem or estimating oriented normals. Yet, our approach results in a rendered image as if a surface mesh was reconstructed. We demonstrate that our framework produces a plausible image, and can effectively handle noise, non-uniform sampling, thin surfaces / sheets, and is fast.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 7

page 8

research
04/24/2023

Pointersect: Neural Rendering with Cloud-Ray Intersection

We propose a novel method that renders point clouds as if they are surfa...
research
09/09/2021

Neural-IMLS: Learning Implicit Moving Least-Squares for Surface Reconstruction from Unoriented Point Clouds

Surface reconstruction from noisy, non-uniform, and unoriented point clo...
research
10/27/2022

Boosting Point Clouds Rendering via Radiance Mapping

Recent years we have witnessed rapid development in NeRF-based image ren...
research
09/08/2023

Dynamic Mesh-Aware Radiance Fields

Embedding polygonal mesh assets within photorealistic Neural Radience Fi...
research
10/19/2020

GAMesh: Guided and Augmented Meshing for Deep Point Networks

We present a new meshing algorithm called guided and augmented meshing, ...
research
07/28/2017

Continuous Global Optimization in Surface Reconstruction from an Oriented Point Cloud

We introduce a continuous global optimization method to the field of sur...
research
04/14/2021

Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases

We propose a method for the unsupervised reconstruction of a temporally-...

Please sign up or login with your details

Forgot password? Click here to reset