Classification of moving objects through scattering media based on a dynamic vision sensor

Conventional methods for object classification in speckle imaging through thin scattering media are predominantly designed for stationary objects. However, moving objects introduce motion blur into speckle images, and diminishing classification accuracy. To address this challenge, we propose to take advantage of event data to enhance images captured by a grayscale camera. We introduce the event-fused transformer (EFT), a transformer-based network designed to detect moving objects through scattering media. The EFT employs the vision transformer (ViT) as its backbone and integrates a spike neural network (SNN)-based transformer module to extract and fuse features from event data. These features effectively improve blurred speckle textures and guide the focus of the network on critical regions during training. To validate our approach, we prepared grayscale and event-based speckle datasets using three widely used datasets. Experimental results demonstrate that the incorporation of event information effectively improves the classification accuracy, increasing it from 94.875% to 97.125% compared to methods that rely solely on blurred grayscale speckle images.