Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks

by   Amy Jin, et al.

Five billion people in the world lack access to quality surgical care. Surgeon skill varies dramatically, and many surgical patients suffer complications and avoidable harm. Improving surgical training and feedback would help to reduce the rate of complications, half of which have been shown to be preventable. To do this, it is essential to assess operative skill, a process that currently requires experts and is manual, time consuming, and subjective. In this work, we introduce an approach to automatically assess surgeon performance by tracking and analyzing tool movements in surgical videos, leveraging region-based convolutional neural networks. In order to study this problem, we also introduce a new dataset, m2cai16-tool-locations, which extends the m2cai16-tool dataset with spatial bounds of tools. While previous methods have addressed tool presence detection, ours is the first to not only detect presence but also spatially localize surgical tools in real-world laparoscopic surgical videos. We show that our method both effectively detects the spatial bounds of tools as well as significantly outperforms existing methods on tool presence detection. We further demonstrate the ability of our method to assess surgical quality through analysis of tool usage patterns, movement range, and economy of motion.


page 1

page 3

page 5

page 7


Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection

Video understanding of robot-assisted surgery (RAS) videos is an active ...

Multi-label Classification of Surgical Tools with Convolutional Neural Networks

Automatic tool detection from surgical imagery has a multitude of useful...

Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos

Purpose: Real-time surgical tool tracking is a core component of the fut...

How Can I Swing Like Pro?: Golf Swing Analysis Tool for Self Training

In this work, we present an analysis tool to help golf beginners compare...

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Surgical tool localization is an essential task for the automatic analys...

Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery

Open, or non-laparoscopic surgery, represents the vast majority of all o...

Deepwound: Automated Postoperative Wound Assessment and Surgical Site Surveillance through Convolutional Neural Networks

Postoperative wound complications are a significant cause of expense for...

Please sign up or login with your details

Forgot password? Click here to reset