Hybrid spectral-spatial domain registration for nanometric tracking in digital in-line holographic microscopy

Digital in-line holographic microscopy enables label-free tracking and metrology, but achieving nanometric sub-pixel displacement accuracy over a wide capture range remains challenging. Frequency-domain registration based on the discrete Fourier transform (DFT) is globally stable and tolerant to large shifts, yet it suffers from sub-pixel quantization and interpolation artifacts that limit precision near zero displacement. In contrast, spatial-gradient refinement such as Lucas-Kanade (LK) can reach very high sub-pixel accuracy, but it is strongly initialization-limited and prone to divergence outside a narrow convergence basin. Here, we propose a Hybrid Spectral-Spatial Domain (HSSD) framework that resolves this trade-off by combining the global robustness of a DFT-based coarse estimator with LK refinement of the residual motion. We validate HSSD using numerical simulations and experiments in a transmission in-line holographic imaging system, achieving nanometric-precision displacement measurement and stable tracking across a wide range of displacements and defocusing conditions. This hybrid strategy enables reliable nanometric localization in regimes where standalone DFT or LK methods either fail to converge or saturate in accuracy.