SpectroFusionNet a CNN approach utilizing spectrogram fusion for electric guitar play recognition

Area of Science:

• Music Information Retrieval
• Deep Learning
• Signal Processing

Background:

• Music is a universal language integral to human expression.
• Automated recognition of musical instruments and techniques is a growing field.
• Electric guitar playing involves complex techniques that are challenging to classify.

Purpose of the Study:

• To introduce SpectroFusionNet, a deep learning framework for automated electric guitar playing technique recognition.
• To explore various spectrogram extraction and feature fusion strategies for improved classification.
• To evaluate the performance of the proposed framework on distinct guitar sound classes.

Main Methods:

• Extraction of Mel-Frequency Cepstral Coefficients (MFCC), Continuous Wavelet Transform (CWT), and Gammatone spectrograms.
• Individual processing of spectrograms using lightweight deep learning models (MobileNetV2, InceptionV3, ResNet50).
• Application of early and late fusion strategies, followed by classification using nine machine learning models (SVM, MLP, Random Forest, etc.).

Main Results:

• The MFCC-Gammatone late fusion strategy achieved the highest performance: 99.12% accuracy, 100% precision, and 100% recall across 9 classes.
• ResNet50 showed better performance in individual spectrogram processing.
• SpectroFusionNet demonstrated real-world applicability with 70.9% accuracy on a real-time audio dataset.

Conclusions:

• SpectroFusionNet effectively automates the recognition of electric guitar playing techniques.
• Late fusion of MFCC and Gammatone spectrograms offers superior feature representation for classification.
• The framework shows potential for real-world applications in music technology and analysis.