StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation

by   Chi Zhang, et al.

The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To address the misalignment between poses and images in data, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. We also delve into attribute-related prompts to increase the diversity of the generated avatars. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.


page 1

page 6

page 7

page 8

page 14


IT3D: Improved Text-to-3D Generation with Explicit View Synthesis

Recent strides in Text-to-3D techniques have been propelled by distillin...

MVDream: Multi-view Diffusion for 3D Generation

We propose MVDream, a multi-view diffusion model that is able to generat...

Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models

The ability to generate diverse 3D articulated head avatars is vital to ...

DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Model

The increasing demand for high-quality 3D content creation has motivated...

CG-NeRF: Conditional Generative Neural Radiance Fields

While recent NeRF-based generative models achieve the generation of dive...

360-Degree Panorama Generation from Few Unregistered NFoV Images

360^∘ panoramas are extensively utilized as environmental light sources ...

Divide Bind Your Attention for Improved Generative Semantic Nursing

Emerging large-scale text-to-image generative models, e.g., Stable Diffu...

Please sign up or login with your details

Forgot password? Click here to reset