IT3D: Improved Text-to-3D Generation with Explicit View Synthesis

by   Yiwen Chen, et al.

Recent strides in Text-to-3D techniques have been propelled by distilling knowledge from powerful large text-to-image diffusion models (LDMs). Nonetheless, existing Text-to-3D approaches often grapple with challenges such as over-saturation, inadequate detailing, and unrealistic outputs. This study presents a novel strategy that leverages explicitly synthesized multi-view images to address these issues. Our approach involves the utilization of image-to-image pipelines, empowered by LDMs, to generate posed high-quality images based on the renderings of coarse 3D models. Although the generated images mostly alleviate the aforementioned issues, challenges such as view inconsistency and significant content variance persist due to the inherent generative nature of large diffusion models, posing extensive difficulties in leveraging these images effectively. To overcome this hurdle, we advocate integrating a discriminator alongside a novel Diffusion-GAN dual training strategy to guide the training of 3D models. For the incorporated discriminator, the synthesized multi-view images are considered real data, while the renderings of the optimized 3D models function as fake data. We conduct a comprehensive set of experiments that demonstrate the effectiveness of our method over baseline approaches.


page 2

page 4

page 6

page 7

page 11


StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation

The recent advancements in image-text diffusion models have stimulated r...

MVDream: Multi-view Diffusion for 3D Generation

We propose MVDream, a multi-view diffusion model that is able to generat...

GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis

Synthesizing high-fidelity complex images from text is challenging. Base...

Improved Vector Quantized Diffusion Models

Vector quantized diffusion (VQ-Diffusion) is a powerful generative model...

360-Degree Panorama Generation from Few Unregistered NFoV Images

360^∘ panoramas are extensively utilized as environmental light sources ...

HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation

In this paper, we study Text-to-3D content generation leveraging 2D diff...

Towards Robust GAN-generated Image Detection: a Multi-view Completion Representation

GAN-generated image detection now becomes the first line of defense agai...

Please sign up or login with your details

Forgot password? Click here to reset