Transferable polychromatic optical encoder for neural networks

Artificial neural networks have fundamentally transformed the field of computer vision, providing unprecedented performance. However, these neural networks for image processing demand substantial computational resources, often hindering real-time operation. In this work, we demonstrate an optical encoder that can perform convolution simultaneously in three color channels during the image capture, effectively implementing several initial convolutional layers of the network. Such an optical encoding results in  ~ 24, 000 × reduction in computational operations, with a state-of-the-art classification accuracy (~73.2%) in free-space optical system. In addition, our analog optical encoder, trained for CIFAR-10 data, can be transferred to the ImageNet subset, High-10, without any modifications, and still exhibits moderate accuracy. The proposed method can decrease total system-level energy more than two orders of magnitude per a single object classification. Our results evidence the potential of hybrid optical/digital computer vision system in which the optical frontend can pre-process an ambient scene to reduce the energy and latency of the whole computer vision system.