Enhancing critical view of safety recognition using vision foundation models and state-space temporal modeling

The Critical View of Safety (CVS) is a clinically mandated criterion for reducing bile duct injury during laparoscopic cholecystectomy; however, its automated recognition from endoscopic video remains challenging due to visual occlusion, depth ambiguity, and the inherently progressive nature of CVS formation across time. This work aims to develop a robust and clinically interpretable framework for automated multi-label CVS recognition from monocular surgical video, without relying on auxiliary segmentation supervision or stereo hardware.

We propose a novel spatio-temporal CVS recognition framework that integrates depth-augmented visual representations with efficient temporal state-space modeling. The approach employs a DINOv2-based vision foundation model extended to RGB-D input for robust self-supervised spatial representation learning, alongside a Swin Transformer for hierarchical and anatomically localized feature encoding. To capture the temporal persistence and geometric consistency required for CVS identification, a depth-aware Spatio-temporal Mamba module is introduced to model long-range temporal dependencies with linear computational complexity. The network produces multi-label predictions corresponding to individual CVS criteria.

Experimental evaluations demonstrate that the proposed RGB-D spatio-temporal architecture consistently outperforms RGB-only and attention-based temporal baselines. The method shows improved robustness to visual occlusion, viewpoint variation, and dataset shift across evaluation scenarios.

The proposed RGB-D spatio-temporal framework demonstrates that integrating weight-inflated foundation model adaptation, hierarchical spatial encoding, and depth-conditioned selective state-space temporal modeling yields reliable, scalable, and clinically interpretable CVS recognition. In contrast to prior video SSM approaches that process homogeneous RGB token sequences, the depth-aware gating and dual-encoder fusion introduced here provide targeted inductive biases suited to the geometric and temporal demands of surgical safety verification.