Self-distillation for surgical action recognition

by   Amine Yamlahi, et al.

Surgical scene understanding is a key prerequisite for contextaware decision support in the operating room. While deep learning-based approaches have already reached or even surpassed human performance in various fields, the task of surgical action recognition remains a major challenge. With this contribution, we are the first to investigate the concept of self-distillation as a means of addressing class imbalance and potential label ambiguity in surgical video analysis. Our proposed method is a heterogeneous ensemble of three models that use Swin Transfomers as backbone and the concepts of self-distillation and multi-task learning as core design choices. According to ablation studies performed with the CholecT45 challenge data via cross-validation, the biggest performance boost is achieved by the usage of soft labels obtained by self-distillation. External validation of our method on an independent test set was achieved by providing a Docker container of our inference model to the challenge organizers. According to their analysis, our method outperforms all other solutions submitted to the latest challenge in the field. Our approach thus shows the potential of self-distillation for becoming an important tool in medical image analysis applications.


Language-based Action Concept Spaces Improve Video Self-Supervised Learning

Recent contrastive language image pre-training has led to learning highl...

Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding

Global and local relational reasoning enable scene understanding models ...

CholecTriplet2021: A benchmark challenge for surgical action triplet recognition

Context-aware decision support in the operating room can foster surgical...

Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach

The remarkable performance of the pre-trained language model (LM) using ...

COMPASS: A Formal Framework and Aggregate Dataset for Generalized Surgical Procedure Modeling

Objective: We propose a formal framework for modeling surgical tasks usi...

Situation Awareness for Automated Surgical Check-listing in AI-Assisted Operating Room

Nowadays, there are more surgical procedures that are being performed us...

Shifted-Windows Transformers for the Detection of Cerebral Aneurysms in Microsurgery

Purpose: Microsurgical Aneurysm Clipping Surgery (MACS) carries a high r...

Please sign up or login with your details

Forgot password? Click here to reset