PIVOTS: Aligning unseen structures using preoperative to intraoperative volume-to-surface registration for liver...

Non-rigid registration is essential for augmented reality-guided laparoscopic liver surgery, as it enables the fusion of preoperative information such as tumor location and vascular structures into the limited intraoperative view, thereby enhancing surgical navigation. A prerequisite is the accurate prediction of intraoperative liver deformation, which remains highly challenging due to factors such as large deformation caused by pneumoperitoneum, respiration and tool interaction as well as noisy intraoperative data, and limited field of view due to occlusion and constrained camera movement. To address these challenges, we introduce PIVOTS, a Preoperative to Intraoperative VOlume-To-Surface registration neural network that directly takes point clouds as input for deformation prediction. The geometric feature extraction encoder allows multi-resolution feature extraction, and the decoder, comprising inter-modality cross attention modules, enables information exchange between pre- and intraoperative features and accurate multi-level displacement prediction. We train the neural network on a large synthetic dataset created using a biomechanical simulation pipeline that explicitly targets the mentioned intraoperative challenges and validate its performance on both synthetic and real datasets. Results demonstrate superior registration performance of our method compared to baseline methods, exhibiting strong robustness against high amounts of noise, large deformation, and various levels of intraoperative visibility. The network is fast enough to run multiple times per second and directly generalizes to new patients without retraining. We publish training and test sets as evaluation benchmarks in an effort to contribute to the development of more robust liver registration methods based on volume-to-surface data. Code, docker container and datasets are available athttps://github.com/pengliu-nct/PIVOTS.