Spatio-Temporal Representation Decoupling and Enhancement for Federated Instrument Segmentation in Surgical Videos.

Area of Science:

• Computer Vision
• Machine Learning
• Medical Imaging

Background:

• Federated Learning (FL) enables collaborative model training across multiple sites without data centralization.
• Limited FL research exists in surgical data science, and existing methods overlook surgical domain specifics like varied backgrounds and instrument similarities.
• Surgical simulators offer efficient large-scale synthetic data generation.

Purpose of the Study:

• To propose a novel Personalized FL scheme, FedST, leveraging surgical domain knowledge for enhanced instrument segmentation.
• To address challenges of diverse anatomical backgrounds and consistent instrument representation in federated surgical settings.
• To improve model generalization and adaptation to different surgical sites.

Main Methods:

• FedST utilizes a Representation Separation and Cooperation (RSC) mechanism for local-site training, decoupling private background encoding.
• Global training optimizes consistent instrument representations, including temporal layers for motion patterns.
• Textual-guided channel selection enhances site-specific feature adaptation.
• Synthesis-based Explicit Representation Quantification (SERQ) uses synthetic data for synchronized global model convergence.

Main Results:

• FedST achieved superior performance on federated sites, improving IoU by 1.84% compared to state-of-the-art methods.
• Demonstrated significant improvement on an out-of-federation site, with a 45.29% increase in IoU.
• A new PFL benchmark was created with five surgical sites and four data types.

Conclusions:

• The proposed FedST scheme effectively enhances surgical instrument segmentation using personalized Federated Learning.
• FedST demonstrates robust generalization capabilities, adapting well to unseen surgical sites and diverse conditions.
• This work provides a valuable contribution to FL in surgical data science, with potential for real-world clinical applications.