ConsisTNet: a spatio-temporal approach for consistent anatomical localization in endoscopic pituitary surgery

Area of Science:

• Neurosurgery
• Medical Imaging
• Artificial Intelligence

Background:

• Automated localization of critical anatomical structures is vital for safety and outcomes in endoscopic pituitary surgery.
• Current deep learning models often lack frame-to-frame consistency, limiting their intraoperative utility.

Purpose of the Study:

• To introduce ConsisTNet, a novel spatio-temporal model to enhance prediction stability and consistency.
• To address frame-to-frame inconsistencies in deep learning-based anatomical localization for pituitary surgery.

Main Methods:

• ConsisTNet utilizes spatio-temporal features from consecutive frames for consistent predictions.
• A semi-supervised strategy with label propagation was employed for pseudo-label generation.
• TensorRT was used for model optimization and acceleration for real-time performance.

Main Results:

• ConsisTNet significantly improved segmentation consistency (4.56-9.45% IoU increase) and landmark detection consistency (43.86% error reduction).
• The accelerated model achieved 202 FPS inference speed with FP16 precision.
• ConsisTNet demonstrated superior consistency over state-of-the-art models.

Conclusions:

• ConsisTNet offers substantial improvements in spatio-temporal consistency for anatomical localization.
• The model provides more stable and reliable real-time surgical assistance in endoscopic pituitary procedures.