Related Experiment Video
Updated: Apr 27, 2026
Author Spotlight: Learning Systematic Bronchoscopy in a Simulation-Base Setting
Published on: June 23, 2023
BronchOpt: vision-based pose optimization with fine-tuned foundation models for accurate bronchoscopy navigation
Hongchao Shu1, Roger D Soberanis-Mukul2, Jiru Xu2
1Johns Hopkins University, Baltimore, Maryland, 21218, USA. hshu4@jhu.edu.
Purpose:
Accurate intra-operative localization of the endoscope tip relative to the anatomy remains a major challenge in bronchoscopy due to respiratory motion, anatomical variability, and CT-to-body divergence, which cause deformation and misalignment between intra-operative views and pre-operative CT. Existing vision-based methods often struggle to generalize across domains and patients, limiting robustness and leaving residual alignment errors. This work aims to establish a generalizable foundation for bronchoscopy navigation through a robust vision-based framework and a new synthetic benchmark dataset that enables standardized evaluation and reproducible development.
Methods:
We propose a vision-based pose optimization framework for frame-wise 2D-3D registration between intra-operative endoscopic views and pre-operative CT anatomy. A fine-tuned modality- and domain-invariant encoder enables direct similarity measurements between real endoscopic RGB images and CT-rendered depth maps, while differentiable rendering refines camera poses through depth consistency. To enhance reproducibility, we introduce the first public synthetic benchmark dataset for bronchoscopy navigation to address the lack of publicly available paired CT-endoscopy data.
Results:
Trained solely on synthetic data distinct from the benchmark, our model attains an average translational error of 2.65 mm and a rotational error of 0.19 rad, demonstrating high localization accuracy and stability. Qualitative results on real patient data further confirm strong cross-domain generalization, achieving consistent frame-wise 2D-3D alignment without domain-specific adaptation.
Conclusion:
The proposed framework achieves robust, domain-invariant bronchoscopy localization through iterative vision-based optimization, offering a scalable solution toward reliable vision-based bronchoscopy localization. The introduced synthetic benchmark dataset provides a valuable resource for standardized evaluation on bronchoscopy navigation.
