CrossModal-associated-SNN: multi-channel spiking neural networks with clustering and associative learning

Biological sensory systems achieve remarkable robustness through efficient cross-modal integration, yet replicating this in artificial spiking neural networks (SNNs) remains a challenge. Inspired by Mayer's multi-channel learning cognitive theory, we present CrossModal-Associated-SNN, a neuro-inspired framework that synergistically integrates visual and auditory information via Spike-Timing-Dependent Plasticity (STDP) clustering and associative learning. The architecture employs a multi-channel, multi-network design for modality-specific processing, followed by a cross-channel complementary strategy that refines decision-making through associative signals. Evaluated on small-sample benchmarks (MNIST3K and Spoken-MNIST3K), the model demonstrates superior generalization and robustness in multi-modal classification. Compared to the single-channel baseline (91% accuracy), the dual-channel dual-network improved visual and auditory recognition to 93% and 84%, respectively, with a fusion accuracy of 94%. The dual-channel triple-network architecture further maximized performance, attaining 96% (visual), 90% (auditory), and a peak 97% cross-modal fusion accuracy. These results suggest that cooperative shallow micro-networks, akin to biological small-neuron ensembles in superficial brain regions, offer a potentially energy-efficient alternative for multimodal tasks processing. CrossModal-Associated-SNN represents a critical step toward mimicking human sensory cognitive integration, offering a biologically plausible solution for energy-efficient, multi-modal intelligent systems.