A Feature Fusion Method with Guided Training for Classification Tasks.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Accurate classification in recognition tasks often requires integrating diverse data types, including visual and numerical information.
• Existing models like Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) excel with specific data types but struggle with multimodal fusion.
• Challenges remain in effectively combining image and numerical features for complex recognition tasks.

Purpose of the Study:

• To develop an effective feature fusion method, termed FGT-Net, for recognition tasks requiring both image and numerical data.
• To improve classification accuracy by synergistically integrating visual features and structured numerical data.
• To optimize the training process for multimodal feature fusion models.

Main Methods:

• A novel architecture, FGT-Net, comprising a shared weight network, a feature fused layer, and a classification layer was proposed.
• A guided training method was introduced to enhance the extraction of image features within the shared weight network.
• Image features and numerical features were fused in the feature fused layer before final classification.

Main Results:

• The FGT-Net achieved a classification accuracy of 87.8%.
• This represents a 15% improvement over ShuffleNetv2 (CNN, image data only).
• The model also showed a 9.8% accuracy increase compared to DNN (structured data only).

Conclusions:

• The proposed FGT-Net effectively fuses image and numerical data, leading to superior recognition performance.
• Guided training significantly enhances the model's ability to learn discriminative features from images.
• FGT-Net offers a robust solution for recognition tasks where multimodal data integration is crucial.