DeepAI AI Chat
Log In Sign Up

A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

by   Dominik J. E. Waibel, et al.

Diffusion models are a class of generative models, showing superior performance as compared to other generative models in creating realistic images when trained on natural image datasets. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy image as a prior, DISPR is conditioned to predict realistic 3D shape reconstructions. To showcase the applicability of DISPR as a data augmentation tool in a feature-based single cell classification task, we extract morphological features from the cells grouped into six highly imbalanced classes. Adding features from predictions of DISPR to the three minority classes improved the macro F1 score from F1_macro = 55.2 ± 4.6% to F1_macro = 72.2 ± 4.9%. With our method being the first to employ a diffusion-based model in this context, we demonstrate that diffusion models can be applied to inverse problems in 3D, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.


page 1

page 2

page 3


MiShape: 3D Shape Modelling of Mitochondria in Microscopy

Fluorescence microscopy is a quintessential tool for observing cells and...

Generative modeling of living cells with SO(3)-equivariant implicit neural representations

Data-driven cell tracking and segmentation methods in biomedical imaging...

3D single-cell shape analysis of cancer cells using geometric deep learning

Aberrations in cell geometry are linked to cell signalling and disease. ...

Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models

Diffusion models have emerged as the new state-of-the-art generative mod...

Implicit Neural Representations for Generative Modeling of Living Cell Shapes

Methods allowing the synthesis of realistic cell shapes could help gener...

An Efficient Approach for Cell Segmentation in Phase Contrast Microscopy Images

In this paper, we propose a new model to segment cells in phase contrast...

Generative Modeling with Conditional Autoencoders: Building an Integrated Cell

We present a conditional generative model to learn variation in cell and...