GenerateCT: Text-Guided 3D Chest CT Generation

by   Ibrahim Ethem Hamamci, et al.
Universität Zürich

Generative modeling has experienced substantial progress in recent years, particularly in text-to-image and text-to-video synthesis. However, the medical field has not yet fully exploited the potential of large-scale foundational models for synthetic data generation. In this paper, we introduce GenerateCT, the first method for text-conditional computed tomography (CT) generation, addressing the limitations in 3D medical imaging research and making our entire framework open-source. GenerateCT consists of a pre-trained large language model, a transformer-based text-conditional 3D chest CT generation architecture, and a text-conditional spatial super-resolution diffusion model. We also propose CT-ViT, which efficiently compresses CT volumes while preserving auto-regressiveness in-depth, enabling the generation of 3D CT volumes with variable numbers of axial slices. Our experiments demonstrate that GenerateCT can produce realistic, high-resolution, and high-fidelity 3D chest CT volumes consistent with medical language text prompts. We further investigate the potential of GenerateCT by training a model using generated CT volumes for multi-abnormality classification of chest CT volumes. Our contributions provide a valuable foundation for future research in text-conditional 3D medical image generation and have the potential to accelerate advancements in medical imaging research. Our code, pre-trained models, and generated data are available at


page 1

page 3

page 7

page 13

page 14


CT-SGAN: Computed Tomography Synthesis GAN

Diversity in data is critical for the successful training of deep learni...

COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that h...

Spot the fake lungs: Generating Synthetic Medical Images using Neural Diffusion Models

Generative models are becoming popular for the synthesis of medical imag...

Zero-shot CT Field-of-view Completion with Unconditional Generative Diffusion Prior

Anatomically consistent field-of-view (FOV) completion to recover trunca...

Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis

While recent advances in large-scale foundational models show promising ...

CT Image Harmonization for Enhancing Radiomics Studies

While remarkable advances have been made in Computed Tomography (CT), ca...

Please sign up or login with your details

Forgot password? Click here to reset