Optical multi-task learning using a multifunctional diffractive processor with "learned" structured illumination

Diffractive deep neural network (D²NN), as a novel optical machine learning framework, has attracted much attention due to its superior advantages in inference speed and power consumption. Here, we demonstrate a new, to the best of our knowledge, framework design for multifunctional D²NN with "learned" structured illumination, incorporating the illumination patterns as trainable parameters into the network optimization process. Compared with the traditional D²NN design, this network framework enables the reuse of diffractive layers while quickly switching between different machine learning tasks, significantly expanding the network's practicality. Based on this, two different training approaches are proposed: one based on transfer learning and the other based on constructing a multi-input and multi-output (MIMO) neural network. The approaches are tested on three different tasks: MNIST dataset classification, Fashion-MNIST dataset classification, and computational imaging. The D²NN framework we proposed not only improves the flexibility of optical neural networks but also paves a new path for the construction of multifunctional optical machine learning platforms.