Meta-Optical Encoder for Image Segmentation.

Area of Science:

• Optics and Photonics
• Artificial Intelligence
• Computer Vision

Background:

• Deep neural networks (DNNs) excel at image segmentation but are computationally demanding for resource-constrained devices.
• Existing compression methods for DNNs often lead to significant accuracy degradation.
• There is a need for efficient neural network architectures suitable for edge computing and autonomous systems.

Purpose of the Study:

• To propose and experimentally demonstrate a hybrid optical/digital neural network for complex image segmentation.
• To significantly reduce the computational complexity and parameter count of deep neural networks for image segmentation.
• To enable practical low-power, high-speed optical neural network systems for edge devices.

Main Methods:

• Integration of a meta-optical encoder for optical domain convolution, reducing computational load.
• Application of knowledge distillation to compress a U-Net architecture into a compact parallel network.
• Replacement of initial digital convolutional layers with an engineered metasurface for optical processing.

Main Results:

• Achieved an approximately 12,000x reduction in parameters and an 800x reduction in operations.
• Maintained a segmentation accuracy of approximately 81.8%, with a minor accuracy loss of ~16.6%.
• Outperformed fully digital compressed models of similar complexity, which showed a ~31.0% accuracy loss.

Conclusions:

• The developed hybrid optical/digital neural network is the first experimental system using incoherent light for complex image segmentation.
• This approach offers a viable pathway towards practical, low-power, high-speed optical neural networks for edge computing.
• The hybrid architecture significantly enhances efficiency while maintaining competitive accuracy for image segmentation tasks.