Dual-TTFNet: An end-to-end dual-branch temporal and time-frequency fusion network for auditory attention decoding in...

Auditory attention decoding based on steady-state motion auditory evoked potential (SSMAEP) offers a promising pathway for developing auditory brain-computer interface (BCI) driven by auditory selective attention. However, achieving high decoding performance with strong interpretability remains a major challenge. To address this issue, we proposed an end-to-end dual-branch neural network that fuses temporal and time-frequency information (Dual-TTFNet) to enhance SSMAEP decoding performance. The model consisted of a temporal convolutional branch and a time-frequency branch with learnable S-transform convolutional kernels for modeling of time-frequency patterns. To further strengthen inter-branch interactions, bidirectional cross-branch EEG channel attention mechanism and attention mechanism-based Transformer was introduced to achieve deep integration of temporal and time-frequency representations. Experiments on two and three-target SSMAEP-BCI datasets demonstrate that Dual-TTFNet consistently outperforms state-of-the-art methods under various tasks, time windows, and EEG channel configurations. It achieved accuracies of 95.08 ± 7.46 % (two-class) and 91.50 ± 4.90 % (three-class) at 5 s, with information transfer rate of 7.94 ± 3.08 bits/min and 11.06 ± 2.35 bits/min, respectively. Ablation studies and visualization analyses further validated the crucial role of the attention mechanisms and S-transform kernels in enhancing feature discriminability and neural interpretability. Dual-TTFNet achieves a synergistic optimization of SSMAEP-BCI decoding performance and interpretability, demonstrating excellent generalization ability and application potential.