Cross-Modality Interaction Network for Equine Activity Recognition Using Imbalanced Multi-Modal Data.

Area of Science:

• Animal behavior analysis
• Machine learning in animal science
• Wearable sensor technology

Background:

• Automated animal activity recognition benefits from deep learning and wearable sensors.
• Key challenges include effective multi-modal feature fusion and imbalanced data modeling.
• Equine activity classification requires robust methods to address these limitations.

Purpose of the Study:

• To develop an advanced deep learning model for enhanced equine activity classification.
• To address challenges in multi-modal feature fusion and imbalanced data.
• To improve the accuracy and reliability of automated recognition of horse behaviors.

Main Methods:

• Developed a cross-modality interaction network (CMI-Net) with a dual convolution neural network and a cross-modality interaction module (CMIM).
• Employed CMIM for adaptive recalibration of temporal and axis-wise features, enabling deep intermodality interaction.
• Utilized class-balanced (CB) focal loss to mitigate the class imbalance problem during CMI-Net training.
• Collected motion data from six horses using neck-attached inertial measurement units.

Main Results:

• CMI-Net achieved high performance metrics: 79.74% precision, 79.57% recall, 79.02% F1-score, and 93.37% accuracy.
• The adoption of CB focal loss further boosted CMI-Net performance, increasing precision by 2.76%, recall by 4.16%, and F1-score by 3.92%.
• CMI-Net demonstrated superior performance compared to existing algorithms in equine activity classification.

Conclusions:

• CMI-Net effectively enhances equine activity classification using imbalanced multi-modal sensor data.
• CB focal loss significantly improves the performance of CMI-Net on imbalanced datasets.
• The developed approach offers a promising solution for accurate and reliable automated recognition of horse activities.