TEXTure: Text-Guided Texturing of 3D Shapes

02/03/2023
by   Elad Richardson, et al.
0

In this paper, we present TEXTure, a novel method for text-guided generation, editing, and transfer of textures for 3D shapes. Leveraging a pretrained depth-to-image diffusion model, TEXTure applies an iterative scheme that paints a 3D model from different viewpoints. Yet, while depth-to-image models can create plausible textures from a single viewpoint, the stochastic nature of the generation process can cause many inconsistencies when texturing an entire 3D object. To tackle these problems, we dynamically define a trimap partitioning of the rendered image into three progression states, and present a novel elaborated diffusion sampling process that uses this trimap representation to generate seamless textures from different views. We then show that one can transfer the generated texture maps to new 3D geometries without requiring explicit surface-to-surface mapping, as well as extract semantic textures from a set of images without requiring any explicit reconstruction. Finally, we show that TEXTure can be used to not only generate new textures but also edit and refine existing textures using either a text prompt or user-provided scribbles. We demonstrate that our TEXTuring method excels at generating, transferring, and editing textures through extensive evaluation, and further close the gap between 2D image generation and 3D texturing.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

page 12

page 13

research
05/10/2023

Text-guided High-definition Consistency Texture Model

With the advent of depth-to-image diffusion models, text-guided generati...
research
05/15/2021

Texture Generation with Neural Cellular Automata

Neural Cellular Automata (NCA) have shown a remarkable ability to learn ...
research
11/13/2015

Procedural wood textures

Existing bidirectional reflectance distribution function (BRDF) models a...
research
08/31/2023

Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details

We propose Text2Scene, a method to automatically create realistic textur...
research
09/09/2022

ISS: Image as Stetting Stone for Text-Guided 3D Shape Generation

Text-guided 3D shape generation remains challenging due to the absence o...
research
03/04/2020

Learning to Transfer Texture from Clothing Images to 3D Humans

In this paper, we present a simple yet effective method to automatically...
research
06/03/2016

Incorporating long-range consistency in CNN-based texture generation

Gatys et al. (2015) showed that pair-wise products of features in a conv...

Please sign up or login with your details

Forgot password? Click here to reset