AUGUR-AI: a real-time artificial intelligence interpreter for indocyanine green fluorescence angiography in...

Assessment of intestinal perfusion sufficiency is a key surgeon judgment regarding anastomotic construction. Recently indocyanine green fluorescence angiography (ICGFA) has been proven as a digital technology that at least matches such acumen when correctly interpreted. Here, we report a deployed artificial intelligence (AI) device that automatically provides and documents expert-ICGFA interpretation in real-time operations.

Intraoperative ICGFA videos from consenting patients undergoing colorectal resection and primary anastomosis without anastomotic leak were used to create, train and test two AI models for (1) ICGFA interpretation (using Long Short-Term Memory classification architecture) and (2) auto-segmentation for colon identification from operative expert annotations. Both were deployed with a zero-touch interface in a commercially available edge device (NVIDIA Jetson). Additional datasets from other surgeons, institutions and imaging systems provided further validation. Training/testing used a randomly selected 80%/20% hold-out split, repeated 5 times and averaged, to yield final results. Intersection of Union (IOU) between predicted and ground truth annotations assessed algorithm performance (IOU =  > 0.5 indicating correct prediction), with cross-entropy loss evaluation estimation for generalisability.

48 patient videos from a single institution provided sufficient ICGFA model training-testing with 3,414 annotated images for segmentation modelling. IOU scores for training and testing of the ICGFA interpretation model were 0.72 and 0.62, respectively, with gradual convergence of cross-entropy training loss. Validation testing of the auto-segmentation model achieved an IOU score of 0.86. Further ICGFA model testing on 15 additional unseen videos returned IOU = 0.45 with, importantly, no prediction when classifying one case of poor perfusion. Complete end-to-end deployment live in-theatre, via direct imaging feed and including automatic or zero-touch interaction and Go-No Go live mask segmentation, in 15 surgeries fitted existing surgical workflows.

This clinically deployed AI device demonstrates sufficient promise to proceed to prospective multicentre validation.