Tumour-focused 3D-2D registration in liver laparoscopy

Augmented reality (AR) is a promising tumour-resection guidance tool in liver laparoscopy. Existing methods register a preoperative liver 3D model (reconstructed from CT or MRI) to 2D laparoscopic images. They use surface landmarks to solve for registration and then transfer and overlay the internal tumour. This has two main weaknesses, hindering clinical adoption. First, computing the registration is highly challenging due to (i) multimodal and multidimensional correspondences (colourless 3D model to 2D RGB images), (ii) restricted field of view (proximity of laparoscope to parenchyma), and (iii) liver deformations (breathing and instrument interactions). Second, extrapolating from the surface to deeper tumours increases uncertainty and may result in inaccurate AR.

We propose tumour-focused 3D-2D registration (Tf32), a method that registers the tumour, not the parenchyma. Tf32 exploits laparoscopic ultrasonography (LUS), which provides intraoperative cross-sections of the tumour and can be reliably localised with respect to the laparoscope. Tf32 matches the LUS tumour profile to a set of tumour profiles simulated preoperatively from the preoperative tumour 3D model. This initial registration is then refined using iterative closest point (ICP). A multiple-hypothesis implementation handles ambiguities and local minima arising from the finite profile sample set.

Tf32 achieved 91.07% success rate in meeting the $10\text{mm}$ oncological margin on semi-synthetic data and established a new state-of-the-art on a public benchmark dataset.

Compared with competitors, Tf32 does not require manual initialisation nor the full liver 3D model and is notably more accurate, making it well-adapted to clinical use.