Intrinsic Neural Fields: Learning Functions on Manifolds

by   Lukas Koestler, et al.
Technische Universität München
University of Siegen

Neural fields have gained significant attention in the computer vision community due to their excellent performance in novel view synthesis, geometry reconstruction, and generative modeling. Some of their advantages are a sound theoretic foundation and an easy implementation in current deep learning frameworks. While neural fields have been applied to signals on manifolds, e.g., for texture reconstruction, their representation has been limited to extrinsically embedding the shape into Euclidean space. The extrinsic embedding ignores known intrinsic manifold properties and is inflexible wrt. transfer of the learned function. To overcome these limitations, this work introduces intrinsic neural fields, a novel and versatile representation for neural fields on manifolds. Intrinsic neural fields combine the advantages of neural fields with the spectral properties of the Laplace-Beltrami operator. We show theoretically that intrinsic neural fields inherit many desirable properties of the extrinsic neural field framework but exhibit additional intrinsic qualities, like isometry invariance. In experiments, we show intrinsic neural fields can reconstruct high-fidelity textures from images with state-of-the-art quality and are robust to the discretization of the underlying manifold. We demonstrate the versatility of intrinsic neural fields by tackling various applications: texture transfer between deformed shapes different shapes, texture reconstruction from real-world images with view dependence, and discretization-agnostic learning on meshes and point clouds.


page 7

page 11

page 14

page 30


Intrinsic and extrinsic deep learning on manifolds

We propose extrinsic and intrinsic deep neural network architectures as ...

Texture Fields: Learning Texture Representations in Function Space

In recent years, substantial progress has been achieved in learning-base...

Leveraging Intrinsic Properties for Non-Rigid Garment Alignment

We address the problem of aligning real-world 3D data of garments, which...

From data to functa: Your data point is a function and you should treat it like one

It is common practice in deep learning to represent a measurement of the...

IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

Recently, many works have been proposed to utilize the neural radiance f...

NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments

The fields of SocialVR, performance capture, and virtual try-on are ofte...

Intrinsic Isometric Manifold Learning with Application to Localization

Data living on manifolds commonly appear in many applications. We show t...

Please sign up or login with your details

Forgot password? Click here to reset