CapMatch: Semi-Supervised Contrastive Transformer Capsule With Feature-Based Knowledge Distillation for Human...

Area of Science:

• Computer Science
• Machine Learning
• Signal Processing

Background:

• Human Activity Recognition (HAR) is crucial for wearable technology.
• Existing semi-supervised learning (SSL) methods for HAR can be complex.
• Need for efficient and accurate HAR with limited labeled data.

Purpose of the Study:

• To propose CapMatch, a simplified semi-supervised contrastive capsule transformer method for HAR.
• To hybridize supervised and unsupervised learning for robust feature extraction.
• To enhance HAR performance using feature-based knowledge distillation (KD).

Main Methods:

• Developed CapMatch, integrating pseudolabeling, contrastive learning (CL), and feature-based KD.
• Utilized weak and timecut data augmentations for similarity learning.
• Designed a capsule transformer network with capsule-based transformer blocks and a routing layer.

Main Results:

• CapMatch achieved >85% accuracy on HAPT, WISDM, and UCI_HAR datasets with only 10% labeled data.
• Outperformed 14 semi-supervised algorithms under low-label conditions.
• Reached ≥88% accuracy with 30% labeled data, surpassing classical supervised algorithms like decision trees and KNN.

Conclusions:

• CapMatch offers a simplified yet effective SSL approach for HAR.
• Demonstrates superior performance in extracting rich representations from limited labeled data.
• Highlights the potential of capsule transformer networks in HAR tasks.