Deep Learning Approach for Pulmonary Orientation Detection: Toward Automated Intraoperative Imaging Guidance

Robot-assisted thoracic surgery (RATS) for anatomical lung resections remains technically demanding due to patient-specific anatomical variation and the absence of tactile feedback. To support intraoperative orientation, we explored a deep learning-based approach for automated video-based recognition of pulmonary anatomical views during RATS.

We developed the first annotated dataset of 27 RATS lobectomy and segmentectomy procedures, labeling 5 primary orientations (anterior, posterior, inferior, superior, fissure) and 3 auxiliary classes (transition, other, nonsurgical). A 2-stage model combining ResNet50 for spatial feature extraction and a causal Multi-Stage Temporal Convolutional Network (MS-TCN) for temporal modeling was trained and evaluated using 5-fold cross-validation. Model performance was assessed using accuracy, precision, recall, and F1 score.

The dataset contained 27 procedures with a mean duration of 129 min. The "fissure view" was the most frequent orientation (42.5%). The model achieved an overall accuracy of 62%, with the highest recall for the fissure view (0.85). Performance was lower for underrepresented orientations.

This proof-of-concept study demonstrates the first technical feasibility of automated intraoperative orientation recognition during RATS using a deep learning framework. The work provides a novel annotated dataset and an exploratory foundation for future research. Further model refinement, dataset expansion, and prospective validation are needed before clinical implementation.