Virtual reality-augmented differentiable simulations for digital twin applications in surgical planning

This paper presents a novel approach for digital twin applications in surgical planning, integrating a differentiable simulator for trajectory generation within segmented medical images and a virtual reality (VR) platform for navigating an overlay of medical images and generated trajectories. The first section of this study delineates a path-planning method utilizing Langevin random walkers subjected to both local and non-local trajectory constraints. This stochastic process effectively navigates the complex topography of medical imaging data without necessitating comprehensive surgical instrument modeling or replicating the environment's physical properties. The second section introduces a custom-developed shader that seamlessly integrates raw and segmented medical images with potential surgical trajectories within a VR environment. This VR integration provides surgeons an immersive and intuitive platform, facilitating interactive exploration and selecting optimal surgical paths. The system extends beyond predefined trajectory generation criteria, allowing real-time adjustments based on the surgeon's expertise and situational assessment. The approach combines these advanced computational techniques with state-of-the-art visualization methods to enhance surgical planning precision and efficiency. This integrated digital twin strategy can potentially improve preoperative decision-making significantly and, consequently, surgical outcomes.