BronchOpt: vision-based pose optimization with fine-tuned foundation models for accurate bronchoscopy navigation.

Area of Science:

• Medical Imaging
• Computer Vision
• Robotics

Background:

• Accurate intra-operative endoscope localization in bronchoscopy is challenging due to respiratory motion and anatomical variability.
• Existing vision-based methods lack robustness and generalizability across patients and domains.
• Misalignment between intra-operative views and pre-operative CT scans hinders effective navigation.

Purpose of the Study:

• To develop a generalizable, vision-based framework for robust bronchoscopy navigation.
• To establish a new synthetic benchmark dataset for standardized evaluation and reproducible development.
• To overcome limitations of current methods in achieving accurate endoscope tip localization.

Main Methods:

• A vision-based pose optimization framework for 2D-3D registration between endoscopic views and CT anatomy.
• A fine-tuned, modality- and domain-invariant encoder for similarity measurements.
• Differentiable rendering for camera pose refinement using depth consistency.
• Introduction of a public synthetic benchmark dataset for bronchoscopy navigation.

Main Results:

• The model achieved an average translational error of 2.65 mm and rotational error of 0.19 rad when trained solely on synthetic data.
• Demonstrated high localization accuracy and stability.
• Showcased strong cross-domain generalization on real patient data, achieving consistent 2D-3D alignment without domain-specific adaptation.

Conclusions:

• The proposed framework offers robust, domain-invariant bronchoscopy localization via iterative vision-based optimization.
• Provides a scalable solution for reliable, vision-based endoscope localization.
• The synthetic benchmark dataset serves as a valuable resource for standardized evaluation in bronchoscopy navigation.