FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos

Area of Science:

• Medical Imaging
• Surgical Technology
• Deep Learning

Background:

• Fetoscopic laser photocoagulation treats twin-to-twin transfusion syndrome (TTTS).
• Challenges include limited field-of-view, occlusions, and low visibility during surgery.
• Computer-assisted techniques can improve anatomical understanding and surgical precision.

Purpose of the Study:

• To develop an automated computer-assisted technique for identifying vascular anastomoses during fetoscopic laser photocoagulation.
• To improve the understanding of anatomical structures for risk-free laser photocoagulation.
• To enhance mosaics from fetoscopic videos for better surgical guidance.

Main Methods:

• Proposed FetNet, a hybrid CNN-LSTM architecture for spatio-temporal event identification.
• Adapted existing CNN for spatial feature extraction and integrated with LSTM for inference.
• Utilized differential learning rates for effective pre-trained CNN weight utilization.

Main Results:

• Evaluated FetNet on 7 in vivo fetoscopic videos (5551s, 138,780 frames) from TTTS cases.
• Performed 7-fold cross-validation to ensure robustness, with each video serving as a test set.
• Achieved superior performance compared to existing CNN-based methods due to spatio-temporal modeling.

Conclusions:

• FetNet demonstrated improved inference by modeling spatio-temporal information.
• Online testing achieved a frame rate of 114 fps on a Tesla V100 GPU.
• The method shows potential for real-time computer-assisted interventions (CAI), automating occlusion and photocoagulation identification.