Decentralized, privacy-preserving surgical video analysis with Swarm Learning

Progress in artificial intelligence-based analysis of surgical videos has been constrained by reliance on manual frame-level annotations rather than patient-level outcomes. In addition, concerns about data privacy restrict the exchange of laparoscopic video data and, thereby, multicenter collaboration.

To address these limitations, we developed a pipeline that integrates weakly supervised deep learning with Swarm Learning, a decentralized machine learning approach that enables collaborative model training without data centralization. We evaluate our pipeline using a newly curated dataset of 397 laparoscopic appendectomy recordings from six international surgical centers. We identified optimal modelling configurations (frame sampling rates and model architectures) and subsequently compared Swarm Learning to single-center and centralized learning across three novel patient-level disease staging tasks: (i) binary detection of perforated appendicitis, (ii) laparoscopic grading of appendicitis, and (iii) histopathologic inflammation grading. In addition, we surveyed participating centers to identify real-world barriers to the clinical implementation of our decentralized learning pipeline for surgical video analysis.

For appendicitis grading tasks, frame sampling at 1.0 frames per second and use of the SurgTempoNet architecture resulted in reliable classification performance, outperforming SurgFrameNet and Multiple Instance Learning. Across all three disease staging tasks, Swarm Learning consistently outperformed single-center training and achieved performance comparable to centralized learning, with stable generalization in external validation. The user survey identified hardware failure and limited integration of the decentralized learning pipeline with electronic patient records as key barriers to the clinical implementation of our decentralized learning pipeline for collaborative surgical video analysis.

Weakly supervised deep learning enables the prediction of patient-level endpoints directly from surgical video data. Swarm Learning facilitates privacy-preserving multicenter collaboration and achieves performance on par with centralized learning, highlighting its potential for advancing clinically relevant, collaborative AI development in surgical video analysis, especially when integrated with patients' electronic health records.