Unsupervised domain adaptation for clinician pose estimation and instance segmentation in the OR

by   Vinkle Srivastav, et al.

The fine-grained localization of clinicians in the operating room (OR) is a key component to design the new generation of OR support systems. Computer vision models for person pixel-based segmentation and body-keypoints detection are needed to better understand the clinical activities and the spatial layout of the OR. This is challenging, not only because OR images are very different from traditional vision datasets, but also because data and annotations are hard to collect and generate in the OR due to privacy concerns. To address these concerns, we first study how joint person pose estimation and instance segmentation can be performed on low resolutions images from 1x to 12x. Second, to address the domain shift and the lack of annotations, we propose a novel unsupervised domain adaptation method, called AdaptOR, to adapt a model from an in-the-wild labeled source domain to a statistically different unlabeled target domain. We propose to exploit explicit geometric constraints on the different augmentations of the unlabeled target domain image to generate accurate pseudo labels, and using these pseudo labels to train the model on high- and low-resolution OR images in a self-training framework. Furthermore, we propose disentangled feature normalization to handle the statistically different source and target domain data. Extensive experimental results with detailed ablation studies on the two OR datasets MVOR+ and TUM-OR-test show the effectiveness of our approach against strongly constructed baselines, especially on the low-resolution privacy-preserving OR images. Finally, we show the generality of our method as a semi-supervised learning (SSL) method on the large-scale COCO dataset, where we achieve comparable results with as few as 1% of labeled supervision against a model trained with 100% labeled supervision.


page 2

page 5

page 7

page 12

page 13

page 14


Energy-constrained Self-training for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to transfer the knowledge on a...

Towards Unsupervised Domain Adaptation for Deep Face Recognition under Privacy Constraints via Federated Learning

Unsupervised domain adaptation has been widely adopted to generalize mod...

Constraining Pseudo-label in Self-training Unsupervised Domain Adaptation with Energy-based Model

Deep learning is usually data starved, and the unsupervised domain adapt...

Camera-Driven Representation Learning for Unsupervised Domain Adaptive Person Re-identification

We present a novel unsupervised domain adaption method for person re-ide...

Bridging the Domain Gap in Satellite Pose Estimation: a Self-Training Approach based on Geometrical Constraints

Recently, unsupervised domain adaptation in satellite pose estimation ha...

DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images

In the biomedical domain, there is an abundance of dense, complex data w...

Please sign up or login with your details

Forgot password? Click here to reset