In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains

by   Ting Cao, et al.

Medical applications have benefited from the rapid advancement in computer vision. For patient monitoring in particular, in-bed human posture estimation provides important health-related metrics with potential value in medical condition assessments. Despite great progress in this domain, it remains a challenging task due to substantial ambiguity during occlusions, and the lack of large corpora of manually labeled data for model training, particularly with domains such as thermal infrared imaging which are privacy-preserving, and thus of great interest. Motivated by the effectiveness of self-supervised methods in learning features directly from data, we propose a multi-modal conditional variational autoencoder (MC-VAE) capable of reconstructing features from missing modalities seen during training. This approach is used with HRNet to enable single modality inference for in-bed pose estimation. Through extensive evaluations, we demonstrate that body positions can be effectively recognized from the available modality, achieving on par results with baseline models that are highly dependent on having access to multiple modes at inference time. The proposed framework supports future research towards self-supervised learning that generates a robust model from a single source, and expects it to generalize over many unknown distributions in clinical environments.


page 1

page 2

page 4


Privacy-Preserving In-Bed Pose Monitoring: A Fusion and Reconstruction Study

Recently, in-bed human pose estimation has attracted the interest of res...

Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

We present a self-supervised learning algorithm for 3D human pose estima...

Self-supervised 3D Human Pose Estimation from a Single Image

We propose a new self-supervised method for predicting 3D human body pos...

Learning Privacy-Preserving Optics for Human Pose Estimation

The widespread use of always-connected digital cameras in our everyday l...

mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors

The ability to estimate 3D human body pose and movement, also known as h...

Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data

6D pose estimation pipelines that rely on RGB-only or RGB-D data show li...

Looking At The Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Psychological distress is a significant and growing issue in society. Au...

Please sign up or login with your details

Forgot password? Click here to reset