Design framework for metasurface optics-based convolutional neural networks.

Area of Science:

• Optics and Photonics
• Computer Science
• Artificial Intelligence

Background:

• Deep learning, particularly Convolutional Neural Networks (CNNs), excels in computer vision but demands significant computational resources.
• Current CNN deployment is challenging in resource-constrained environments due to high energy consumption and hardware requirements.
• Existing optical computing methods for CNNs are often limited to grayscale inputs.

Purpose of the Study:

• To propose an end-to-end framework for optically computing CNNs in free-space, akin to a computational camera.
• To develop the first general free-space optics approach capable of processing RGB input data for CNNs.
• To achieve substantial energy savings and sensor simplification without significant loss in network accuracy.

Main Methods:

• Developed a novel framework for free-space optical computation of CNNs.
• Utilized nanoscale metasurface optics to enable processing of multi-channel (RGB) data.
• Designed a system integrating optical components for end-to-end CNN computation.

Main Results:

• The proposed system demonstrates up to an order of magnitude in energy savings compared to conventional methods.
• The optical CNN framework successfully processes RGB input data.
• Network accuracy was largely preserved despite the optical computation approach.

Conclusions:

• Free-space optical computing using metasurface optics offers a viable and energy-efficient solution for deploying CNNs.
• This approach overcomes limitations of previous optical methods by handling RGB data.
• The framework has the potential to revolutionize deep learning deployment in power-limited applications.