Multiple instance convolutional neural network for gallbladder assessment from laparoscopic images

We present an artificial intelligence framework for vascularity classification of the gallbladder (GB) wall from intraoperative images of laparoscopic cholecystectomy (LC).

A two-stage Multiple Instance Convolutional Neural Network is proposed. First, a convolutional autoencoder is trained to extract feature representations from 4585 patches of GB images. The second model includes a multi-instance encoder that fetches random patches from a GB region and outputs an equal number of embeddings that feed a multi-input classification module, which employs pooling and self-attention mechanisms, to perform prediction.

The evaluation was performed on 234 GB images of low and high vascularity from 68 LC videos. Thorough comparison with various state-of-the-art multi-instance and single-instance learning algorithms was performed for two experimental tasks: image- and video-level classification. The proposed framework shows the best performance with accuracy 92.6%-93.2% and F1 93.5%-93.9%, close to the agreement of two expert evaluators (94%).

The proposed technique provides a novel approach to classify LC operations with respect to the vascular pattern of the GB wall.