Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks

by   Gaurav Yengera, et al.
Université de Strasbourg

Real-time algorithms for automatically recognizing surgical phases are needed to develop systems that can provide assistance to surgeons, enable better management of operating room (OR) resources and consequently improve safety within the OR. State-of-the-art surgical phase recognition algorithms using laparoscopic videos are based on fully supervised training. This limits their potential for widespread application, since creation of manual annotations is an expensive process considering the numerous types of existing surgeries and the vast amount of laparoscopic videos available. In this work, we propose a new self-supervised pre-training approach based on the prediction of remaining surgery duration (RSD) from laparoscopic videos. The RSD prediction task is used to pre-train a convolutional neural network (CNN) and long short-term memory (LSTM) network in an end-to-end manner. Our proposed approach utilizes all available data and reduces the reliance on annotated data, thereby facilitating the scaling up of surgical phase recognition algorithms to different kinds of surgeries. Additionally, we present EndoN2N, an end-to-end trained CNN-LSTM model for surgical phase recognition and evaluate the performance of our approach on a dataset of 120 Cholecystectomy laparoscopic videos (Cholec120). This work also presents the first systematic study of self-supervised pre-training approaches to understand the amount of annotations required for surgical phase recognition. Interestingly, the proposed RSD pre-training approach leads to performance improvement even when all the training data is manually annotated and outperforms the single pre-training approach for surgical phase recognition presently published in the literature. It is also observed that end-to-end training of CNN-LSTM networks boosts surgical phase recognition performance.


Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition

Vision algorithms capable of interpreting scenes from a real-time video ...

"Train one, Classify one, Teach one" – Cross-surgery transfer learning for surgical step recognition

Prior work demonstrated the ability of machine learning to automatically...

Rethinking Generalization Performance of Surgical Phase Recognition with Expert-Generated Annotations

As the area of application of deep neural networks expands to areas requ...

RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations

Objective: Accurate surgery duration estimation is necessary for optimal...

LensID: A CNN-RNN-Based Framework Towards Lens Irregularity Detection in Cataract Surgery Videos

A critical complication after cataract surgery is the dislocation of the...

A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning

Virtual orthognathic surgical planning involves simulating surgical corr...

Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos

Purpose: Real-time surgical tool tracking is a core component of the fut...

Please sign up or login with your details

Forgot password? Click here to reset