Physical twinning for joint encoding-decoding optimization in computational optics: a review

Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension, low light throughput, low resolution, and so on. The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine, astronomy, agriculture, etc. With the great advance of artificial intelligence in the last decade, deep learning has further boosted computational optics with higher precision and efficiency. Recently, there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers, and then facilitates simultaneous optimization with the decoding process. This framework offers effective performance enhancement over conventional techniques. However, the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge, due to the discrepant gap in such as bit depth, numerical range, and stability. In this regard, this review explores various optical modulation elements across spatial, phase, and spectral dimensions in the digital twin model for joint encoding-decoding optimization. Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision, speed, and robustness. The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.