Dual-path transformer-based network with equalization-generation components prediction for flexible vibrational...

Area of Science:

• Wearable technology
• Speech processing
• Signal processing

Background:

• Flexible vibrational sensors (FVS) show promise for wearable communication due to inherent noise shielding and soft materials.
• A significant challenge with FVS is the substantial loss of speech frequency components, degrading overall speech quality.
• Existing methods struggle to effectively recover these lost components, limiting FVS adoption in communication.

Purpose of the Study:

• To propose a novel time-domain neural network model, DPT-EGNet, for enhancing the quality of speech captured by flexible vibrational sensors.
• To address the severe loss of frequency components in FVS speech and improve its intelligibility and perceptual quality.
• To develop a model that effectively simulates the inversion process of speech distortion inherent in FVS technology.

Main Methods:

• A dual-path transformer combined with equalization-generation components prediction (DPT-EGNet) model was developed, comprising five modules: pre-processing, dual-path transformer, equalization, generation, and post-processing.
• The dual-path transformer module was utilized to capture both local and global contextual relationships within long-term speech sequences, aiding in the inference of missing information.
• The equalization and generation modules were specifically designed to counteract FVS speech distortion characteristics by simulating an inverse process.

Main Results:

• The DPT-EGNet model demonstrated significant improvements in FVS speech quality, with average increases in Perceptual Evaluation of Speech Quality (PESQ), Short-Time Objective Intelligibility (STOI), and Composite Measure for Overall Speech Quality (COVL) scores by 64.19%, 29.63%, and 101.37%, respectively.
• The proposed model outperformed several baseline models across different domains in enhancing speech quality metrics.
• DPT-EGNet exhibited considerably lower computational complexity compared to other evaluated models, suggesting practical efficiency for real-time applications.

Conclusions:

• The DPT-EGNet model effectively enhances the quality of speech captured by flexible vibrational sensors, overcoming limitations of component loss.
• The proposed neural network architecture provides a superior solution for FVS speech enhancement compared to existing methods.
• The model's efficiency and performance make it a viable candidate for practical wearable communication systems utilizing FVS technology.