Markerless augmented reality registration for surgical guidance: a multi-anatomy clinical accuracy study

In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted display, and assess accuracy across small, or low-curvature anatomies in real-life operative settings.

In HoloLens 2, we align Articulated HAnd Tracking (AHAT) depth to Computed Tomography (CT)-derived skin meshes via (i) depth-bias correction, (ii) brief human-in-the-loop initialization, (iii) global and local registration. We validated the surface-tracing error metric by comparing "skin-to-bone" relative distances to CT ground truth on leg and foot models, using an AR tracked tool. We then performed seven intraoperative target trials (feet $\times$ 2, ear $\times$ 3, leg $\times$ 2) during the initial stage of fibula free-flap harvest and mandibular reconstruction surgery, and collected 500+ data per trial.

Preclinical validation showed tight agreement between AR-traced and CT distances (leg: median $\left|\Delta d\right|$ 0.78 mm, RMSE 0.97 mm; feet: 0.80 mm, 1.20 mm). Clinically, per-point error had a median 3.9 mm. Median errors by anatomy were 3.2 mm (feet), 4.3 mm (ear), and 5.3 mm (lower leg), with 5 mm coverage 92-95%, 84-90%, and 72-86%, respectively. Feet vs. lower leg differed significantly ( $\Delta$ median $\approx$ 1.1 mm; $p<0.001$ ) CONCLUSIONS: A depth-only, markerless AR pipeline on HMDs achieved $\sim$ 3-4 mm median error across feet, ear, and lower leg in live surgical settings without fiducials, approaching typical clinical error thresholds for moderate-risk tasks. Human-guided initialization plus global-to-local registration enabled accurate alignment on small or low-curvature targets , improving the clinical readiness of markerless AR guidance.