A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

08/30/2022
by   Dominik J. E. Waibel, et al.
16

Diffusion models are a class of generative models, showing superior performance as compared to other generative models in creating realistic images when trained on natural image datasets. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy image as a prior, DISPR is conditioned to predict realistic 3D shape reconstructions. To showcase the applicability of DISPR as a data augmentation tool in a feature-based single cell classification task, we extract morphological features from the cells grouped into six highly imbalanced classes. Adding features from predictions of DISPR to the three minority classes improved the macro F1 score from F1_macro = 55.2 ± 4.6% to F1_macro = 72.2 ± 4.9%. With our method being the first to employ a diffusion-based model in this context, we demonstrate that diffusion models can be applied to inverse problems in 3D, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.

READ FULL TEXT

page 1

page 2

page 3

research
03/02/2023

MiShape: 3D Shape Modelling of Mitochondria in Microscopy

Fluorescence microscopy is a quintessential tool for observing cells and...
research
08/13/2023

Shape-guided Conditional Latent Diffusion Models for Synthesising Brain Vasculature

The Circle of Willis (CoW) is the part of cerebral vasculature responsib...
research
04/18/2023

Generative modeling of living cells with SO(3)-equivariant implicit neural representations

Data-driven cell tracking and segmentation methods in biomedical imaging...
research
08/05/2022

3D single-cell shape analysis of cancer cells using geometric deep learning

Aberrations in cell geometry are linked to cell signalling and disease. ...
research
07/04/2023

Learning to reconstruct the bubble distribution with conductivity maps using Invertible Neural Networks and Error Diffusion

Electrolysis is crucial for eco-friendly hydrogen production, but gas bu...
research
06/13/2023

Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data

We present Viewset Diffusion: a framework for training image-conditioned...
research
03/15/2023

Class-Guided Image-to-Image Diffusion: Cell Painting from Brightfield Images with Class Labels

Image-to-image reconstruction problems with free or inexpensive metadata...

Please sign up or login with your details

Forgot password? Click here to reset