A data-driven approach for real-time soft tissue deformation prediction using nonlinear presurgical simulations

A method that allows a fast and accurate registration of digital tissue models obtained during preoperative, diagnostic imaging with those captured intraoperatively using lower-fidelity ultrasound imaging techniques is presented. Minimally invasive surgeries are often planned using preoperative, high-fidelity medical imaging techniques such as MRI and CT imaging. While these techniques allow clinicians to obtain detailed 3D models of the surgical region of interest (ROI), various factors such as physical changes to the tissue, changes in the body's configuration, or apparatus used during the surgery may cause large, non-linear deformations of the ROI. Such deformations of the tissue can result in a severe mismatch between the preoperatively obtained 3D model and the real-time image data acquired during surgery, potentially compromising surgical success. To overcome this challenge, this work presents a new approach for predicting intraoperative soft tissue deformations. The approach works by simply tracking the displacements of a handful of fiducial markers or analogous biological features embedded in the tissue, and produces a 3D deformed version of the high-fidelity ROI model that registers accurately with the intraoperative data. In an offline setting, we use the finite element method to generate deformation fields given various boundary conditions that mimic the realistic environment of soft tissues during a surgery. To reduce the dimensionality of the 3D deformation field involving thousands of degrees of freedom, we use an autoencoder neural network to encode each computed deformation field into a short latent space representation, such that a neural network can accurately map the fiducial marker displacements to the latent space. Our computational tests on a head and neck tumor, a kidney, and an aorta model show prediction errors as small as 0.5 mm. Considering that the typical resolution of interventional ultrasound is around 1 mm and each prediction takes less than 0.5 s, the proposed approach has the potential to be clinically relevant for an accurate tracking of soft tissue deformations during image-guided surgeries.