Introducing surgical workflow recognition in orthopaedic surgery with timestamp supervision

Surgical workflow recognition (SWR) is associated with numerous potential applications to improve patient safety and surgeon performance. So far, SWR studies have mainly focused on endoscopic procedures due to the scarcity of publicly available open surgery video datasets. In this article, we propose for the first time to work on an open orthopaedic surgery called minimally invasive plate osteosynthesis (MIPO) for distal radius fractures (DRFs). For this purpose, we introduce a new dataset with 50 videos of the DRF MIPO procedure, representing almost 30 h of surgery. As far as we know, this currently constitutes the largest publicly available dataset of open surgery videos for SWR. This problem and dataset are particularly challenging due to some specific issues associated with open surgery videos (occlusions, improper camera placements, domain variations, etc.). Using this dataset, we evaluate several state-of-the-art fully-supervised methods to establish a baseline for the prediction of MIPO surgical steps. Then, to reduce the annotation burden required to train such fully-supervised models, we propose a novel weakly-supervised method for SWR, called Uncertainty- and Cluster-Aware Temporal Diffusion (UCATD). UCATD generates reliable pseudo-labels from timestamp annotations (i.e. a single annotated frame per class occurrence) by leveraging uncertainty estimation and clustering information. UCATD significantly outperforms previous state-of-the-art methods based on timestamp supervision, and achieves competitive performance compared to fully-supervised baseline methods, while requiring only 0.1% of the dataset to be annotated.