Classification of moving objects through scattering media based on a dynamic vision sensor.

Area of Science:

• Optics and Photonics
• Computer Vision
• Machine Learning

Background:

• Conventional object classification in scattering media struggles with moving objects due to motion blur.
• Existing methods exhibit diminished classification accuracy when dealing with dynamic scenes.

Purpose of the Study:

• To enhance object classification accuracy in scattering media for moving objects.
• To leverage event data to improve image quality and classification performance.

Main Methods:

• Introduction of the event-fused transformer (EFT), a transformer-based network.
• Integration of a vision transformer (ViT) backbone with a spike neural network (SNN)-based transformer module.
• Fusion of event data features with grayscale speckle images to improve texture and focus.

Main Results:

• Experimental validation using prepared grayscale and event-based speckle datasets.
• Demonstrated improvement in classification accuracy from 94.875% to 97.125% with event data integration.
• Event information effectively enhances blurred speckle textures and guides network focus.

Conclusions:

• The event-fused transformer (EFT) significantly improves moving object classification through scattering media.
• Fusion of event data offers a robust solution for motion blur challenges in speckle imaging.
• The proposed method shows superior performance compared to solely grayscale-based approaches.