An Mcformer encoder integrating Mamba and Cgmlp for improved acoustic feature extraction

Area of Science:

• Artificial Intelligence
• Computer Science
• Signal Processing

Background:

• Conformer architecture is mainstream in speech recognition, integrating self-attention and convolutional networks.
• Conformers have limitations in capturing global information, hindering speech recognition performance.
• Existing models require further optimization for comprehensive feature extraction.

Purpose of the Study:

• Introduce the Mcformer encoder to improve speech recognition capabilities.
• Enhance global context processing and local feature extraction in speech recognition models.
• Address the limitations of current attention models in capturing comprehensive contextual information.

Main Methods:

• The Mcformer encoder incorporates the Mamba module in parallel with multi-head attention blocks.
• A Convolutional Gated Multilayer Perceptron (Cgmlp) structure is employed for improved local feature extraction.
• The model was evaluated on Aishell-1, Common Voice zh 14, and TED-LIUM 3 datasets.

Main Results:

• Mcformer achieved character error rates (CER) of 4.15% and 4.48% on Aishell-1 validation and test sets, respectively (without language model).
• Mcformer achieved CER of 13.28% and 13.06% on Common Voice zh 14 validation and test sets, respectively (without language model).
• On the TED-LIUM 3 dataset, Mcformer achieved word error rates (WER) of 7.26% and 6.95% on validation and test sets, respectively (without language model).

Conclusions:

• The Mcformer encoder demonstrates significant improvements in speech recognition accuracy.
• The integration of Mamba modules and Cgmlp structures effectively enhances both global and local feature processing.
• Experimental results validate the efficacy of Mcformer in reducing error rates across different datasets and languages.