Using neural networks to autonomously assess adequacy in intraoperative cholangiograms

Intraoperative cholangiography (IOC) is a contrast-enhanced X-ray acquired during laparoscopic cholecystectomy. IOC images the biliary tree whereby filling defects, anatomical anomalies and duct injuries can be identified. In Australia, IOC are performed in over 81% of cholecystectomies compared with 20 to 30% internationally (Welfare AIoHa in Australian Atlas of Healthcare Variation, 2017). In this study, we aim to train artificial intelligence (AI) algorithms to interpret anatomy and recognise abnormalities in IOC images. This has potential utility in (a) intraoperative safety mechanisms to limit the risk of missed ductal injury or stone, (b) surgical training and coaching, and (c) auditing of cholangiogram quality.

Semantic segmentation masks were applied to a dataset of 1000 cholangiograms with 10 classes. Classes corresponded to anatomy, filling defects and the cholangiogram catheter instrument. Segmentation masks were applied by a surgical trainee and reviewed by a radiologist. Two convolutional neural networks (CNNs), DeeplabV3+ and U-Net, were trained and validated using 900 (90%) labelled frames. Testing was conducted on 100 (10%) hold-out frames. CNN generated segmentation class masks were compared with ground truth segmentation masks to evaluate performance according to a pixel-wise comparison.

The trained CNNs recognised all classes.. U-Net and DeeplabV3+ achieved a mean F1 of 0.64 and 0.70 respectively in class segmentation, excluding the background class. The presence of individual classes was correctly recognised in over 80% of cases. Given the limited local dataset, these results provide proof of concept in the development of an accurate and clinically useful tool to aid in the interpretation and quality control of intraoperative cholangiograms.

Our results demonstrate that a CNN can be trained to identify anatomical structures in IOC images. Future performance can be improved with the use of larger, more diverse training datasets. Implementation of this technology may provide cholangiogram quality control and improve intraoperative detection of ductal injuries or ductal injuries.