Optimal single-mode squeezing for beam displacement sensing

Estimation of an optical beam's transverse displacement is a canonical imaging problem fundamental to numerous optical imaging and sensing tasks. Quantum enhancements to the measurement precision in this problem have been studied extensively. However, previous studies have neither accounted for diffraction loss in full generality, nor have they addressed how to jointly optimize the spatial mode and the balance between squeezing and coherent amplitude. Here we show that, in the small-displacement limit, the seemingly intractable infinite-spatial-mode problem can be reduced to a compact three-mode interaction framework. We quantify the improvement afforded by an optimized single-spatial-mode Gaussian-state probe over the optimal classical laser probe, and show that a two-spatial-mode homodyne receiver is asymptotically optimal for the former in the limit of high probe energy. Our findings reveal a strategy for identifying quantum-optimal probes in the presence of generic multimode linear probe-target interaction and photon loss.