Multi-scale chunked residual encoding and temporal stochastic interpolation padding in SNNs for enhanced speech classification

Due to the time sensitivity of Spiking Neural Networks (SNNs), they exhibit unique potential for temporal information processing of speech datasets. When they are used to process speech information, the encoding layer is crucial for model's input information and computation. However, the existing SNN encoding mechanism needs to be further optimized in order to effectively extract multi-scale information from speech. Additionally, mismatches in temporal dimensions during speech information processing restrict the application of residual connections in spiking architectures. To address these two issues, we propose the Multi-Scale Chunked Residual Encoder (MCRE) and the Temporal Stochastic Interpolation Padding (TSIP). The execution process of MCRE resembles the information reorganization mechanism of the hippocampus-cortex, enabling parallel processing of both local and global temporal features, effectively integrating these two types of information to enhance the model's contextual understanding. TSIP utilizes a Gaussian distribution to randomly select padding positions along the temporal dimension, thereby achieving equal sequence lengths and enabling the application of residual connections. Furthermore, the introduction of structured noise through the Gaussian distribution can improve the robustness of the model and enhance its overall performance. We have evaluated our method on the Spiking Heidelberg Digits (SHD), Spiking Speech Commands (SSC), and Google Speech Commands v0.02 (GSC) datasets. Our method achieves state-of-the-art (SOTA) results 96.44% accuracy (+1.34%) on SHD and also improves performance on SSC (80.92%, +0.63%) and GSC (95.64%, +0.29%). At the same time, our method significantly reduces the baselines' energy consumption, achieving a maximum reduction of 55%.