Multimodal machine learning for staging laparoscopy: a combined image analysis and morphologic tool for the...

Staging laparoscopy (SL) is an essential procedure for peritoneal metastasis (PM) detection. Although surgeons are expected to differentiate between benign and malignant lesions intraoperatively, this task remains difficult and error-prone. The aim of this study was to develop a novel multimodal machine learning (MML) model to differentiate PM from benign lesions by integrating morphologic characteristics with intraoperative SL images.

Deep learning (DL) models were trained to classify peritoneal lesions in video frames of patients undergoing SL for suspected PM. Two expert surgeons blinded to the pathology results performed an objective morphologic evaluation of these lesions. Traditional machine learning (ML) models were trained to predict tumors based on their morphology. A combined MML model was developed by integrating the best-performing morphology- and image-based models. The MML model was evaluated using an independent test set, and its predictions were compared with those of 13 oncologic surgeons.

The cohort included videos of 67 patients, with 453 consecutive biopsied lesions (benign: n = 197; malignant: n = 256). The MML model achieved an area under the curve (AUC) of 0.88 [95% confidence interval (CI), 0.77-0.96], outperforming the best image-based DL model [AUC = 0.72 (95% CI, 0.54-0.87)], the best morphology-based ML [AUC = 0.86 (95% CI, 0.71-0.95)], and the surgeons' predictions [AUC = 0.78 (95% CI, 0.53-1.00)].

A novel MML model combining the visual and morphologic characteristics of peritoneal lesions was developed and internally validated, demonstrating good discriminative power for classifying PM during SL. This model shows promise as an intraoperative decision-support tool for surgeons, enhancing PM recognition and potentially reducing unnecessary biopsies.