Articulated Clinician Detection Using 3D Pictorial Structures on RGB-D Data

Reliable human pose estimation (HPE) is essential to many clinical applications, such as surgical workflow analysis, radiation safety monitoring and human-robot cooperation. Proposed methods for the operating room (OR) rely either on foreground estimation using a multi-camera system, which is a challenge in real ORs due to color similarities and frequent illumination changes, or on wearable sensors or markers, which are invasive and therefore difficult to introduce in the room. Instead, we propose a novel approach based on Pictorial Structures (PS) and on RGB-D data, which can be easily deployed in real ORs. We extend the PS framework in two ways. First, we build robust and discriminative part detectors using both color and depth images. We also present a novel descriptor for depth images, called histogram of depth differences (HDD). Second, we extend PS to 3D by proposing 3D pairwise constraints and a new method that makes exact inference tractable. Our approach is evaluated for pose estimation and clinician detection on a challenging RGB-D dataset recorded in a busy operating room during live surgeries. We conduct series of experiments to study the different part detectors in conjunction with the various 2D or 3D pairwise constraints. Our comparisons demonstrate that 3D PS with RGB-D part detectors significantly improves the results in a visually challenging operating environment.


page 2

page 3

page 7

page 8


MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation

Person detection and pose estimation is a key requirement to develop int...

3D Human Pose Estimation in RGBD Images for Robotic Task Learning

We propose an approach to estimate 3D human pose in real world units fro...

3D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projections

3D human pose estimation in multi-view operating room (OR) videos is a r...

Face Detection in the Operating Room: Comparison of State-of-the-art Methods and a Self-supervised Approach

Purpose: Face detection is a needed component for the automatic analysis...

Real-time Convolutional Networks for Depth-based Human Pose Estimation

We propose to combine recent Convolutional Neural Networks (CNN) models ...

6D Pose Estimation with Correlation Fusion

6D object pose estimation is widely applied in robotic tasks such as gra...

mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection

This work introduces a new multispectral database and novel approaches f...

Please sign up or login with your details

Forgot password? Click here to reset