CLEAR: Multimodal Human Activity Recognition via Contrastive Learning Based Feature Extraction Refinement

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Human Activity Recognition (HAR) is vital for healthcare and surveillance.
• Sensor-based HAR using AI and IoT shows great promise.
• Ensuring model generalization to new data is a key challenge.

Purpose of the Study:

• To introduce the CLEAR method for improving multimodal human activity recognition accuracy.
• To enhance model generalization capabilities on unseen data.
• To enable direct application of the model to various domains without fine-tuning.

Main Methods:

• Employed data augmentation in time and frequency domains to enrich training data.
• Utilized attention-based multimodal feature fusion for optimized feature extraction.
• Applied supervised contrastive learning to improve feature discriminability.

Main Results:

• Achieved high accuracy rates: 81.09% (USC-HAD), 90.45% (DSADS), and 82.75% (PAMAP2).
• Demonstrated strong generalization: accuracy decreased by only ~5% when training data reduced to 20%.
• CLEAR method showed high performance on unknown datasets using only training data.

Conclusions:

• The CLEAR method significantly enhances multimodal HAR accuracy and generalization.
• The approach effectively extracts discriminative features through augmentation, fusion, and contrastive learning.
• CLEAR offers a robust solution for HAR applications, adaptable to new domains.