Next-generation Surgical Navigation: Multi-view Marker-less 6DoF Pose Estimation of Surgical Instruments

by   Jonas Hein, et al.

State-of-the-art research of traditional computer vision is increasingly leveraged in the surgical domain. A particular focus in computer-assisted surgery is to replace marker-based tracking systems for instrument localization with pure image-based 6DoF pose estimation. However, the state of the art has not yet met the accuracy required for surgical navigation. In this context, we propose a high-fidelity marker-less optical tracking system for surgical instrument localization. We developed a multi-view camera setup consisting of static and mobile cameras and collected a large-scale RGB-D video dataset with dedicated synchronization and data fusions methods. Different state-of-the-art pose estimation methods were integrated into a deep learning pipeline and evaluated on multiple camera configurations. Furthermore, the performance impacts of different input modalities and camera positions, as well as training on purely synthetic data, were compared. The best model achieved an average position and orientation error of 1.3 mm and 1.0 for a surgical drill as well as 3.8 mm and 5.2 for a screwdriver. These results significantly outperform related methods in the literature and are close to clinical-grade accuracy, demonstrating that marker-less tracking of surgical instruments is becoming a feasible alternative to existing marker-based systems.


page 2

page 7


i3PosNet: Instrument Pose Estimation from X-Ray

Performing delicate Minimally Invasive Surgeries (MIS) forces surgeons t...

HMD-EgoPose: Head-Mounted Display-Based Egocentric Marker-Less Tool and Hand Pose Estimation for Augmented Surgical Guidance

The success or failure of modern computer-assisted surgery procedures hi...

Concurrent Segmentation and Localization for Tracking of Surgical Instruments

Real-time instrument tracking is a crucial requirement for various compu...

Occlusion-robust Visual Markerless Bone Tracking for Computer-Assisted Orthopaedic Surgery

Conventional computer-assisted orthopaedic navigation systems rely on th...

Effect of Timing Error: A Case Study of Navigation Camera

We focus on the problem of timing errors in navigation camera as a case ...

MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

Surgical robots are controlled using a priori models based on robots' ge...

Fast and Robust Localization of Surgical Array using Kalman Filter

Intraoperative tracking of surgical instruments is an inevitable task of...

Please sign up or login with your details

Forgot password? Click here to reset