HARE: Unifying the Human Activity Recognition Engineering Workflow

Area of Science:

• Biomedical Engineering
• Computer Science
• Artificial Intelligence

Background:

• Sensor-based human activity recognition (HAR) is crucial in healthcare for monitoring movements.
• Current HAR pipelines are fragmented, involving separate data collection, preparation, and processing steps.
• Compact sensors are increasingly available, driving the need for efficient HAR systems.

Purpose of the Study:

• To present HARE, a comprehensive framework integrating HAR pipeline steps.
• To enable synchronized data collection, labeling, and anonymized pose estimation.
• To introduce real-time HAR with on-device model adaptation and optimal sensor placement.

Main Methods:

• Developed a unified framework (HARE) for HAR.
• Integrated synchronized data collection/labeling and pose estimation for anonymization.
• Implemented a multimodal classification approach with on-device model adaptation.
• Proposed a vision-based method for optimal sensor placement.

Main Results:

• HARE's multimodal, on-device trained model significantly outperformed conventional single-modal and offline methods.
• The vision-based sensor placement approach achieved results comparable to the trained model.
• Extensive evaluations on diverse datasets, including nursing activities, validated the framework's effectiveness.

Conclusions:

• HARE offers a streamlined and integrated solution for sensor-based human activity recognition.
• The framework enhances classification accuracy through multimodal data and on-device learning.
• Novel sensor placement optimization advances the practical application of HAR systems.