Co-occurrence balanced time series classification for the semi-supervised recognition of surgical smoke

Automatic recognition and removal of smoke in surgical procedures can reduce risks to the patient by supporting the surgeon. Surgical smoke changes its visibility over time, impacting the vision depending on its amount and the volume of the body cavity. While modern deep learning algorithms for computer vision require large amounts of data, annotations for training are scarce. This paper investigates the use of unlabeled training data with a modern time-based deep learning algorithm.

We propose to improve the state of the art in smoke recognition by enhancing a image classifier based on convolutional neural networks with a recurrent architecture thereby providing temporal context to the algorithm. We enrich the training with unlabeled recordings from similar procedures. The influence of surgical tools on the smoke recognition task is studied to reduce a possible bias.

The evaluations show that smoke recognition benefits from the additional temporal information during training. The use of unlabeled data from the same domain in a semi-supervised training procedure shows additional improvements reaching an accuracy of 86.8%. The proposed balancing policy is shown to have a positive impact on learning the discrimination of co-occurring surgical tools.

This study presents, to the best of our knowledge, the first use of a time series algorithm for the recognition of surgical smoke and the first use of this algorithm in the described semi-supervised setting. We show that the performance improvements with unlabeled data can be enhanced by integrating temporal context. We also show that adaption of the data distribution is beneficial to avoid learning biases.