How Validation Methodology Influences Human Activity Recognition Mobile Systems.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Human Activity Recognition (HAR) systems are crucial for mobile applications.
• Assessing HAR model performance relies heavily on validation strategies.
• Potential biases in validation can lead to inaccurate performance estimations.

Purpose of the Study:

• To introduce explainable methods for understanding HAR system performance based on validation strategies.
• To identify potential biases introduced by inappropriate validation methodologies.
• To demonstrate the utility of the SHAP framework for HAR model interpretability.

Main Methods:

• Utilized the SHAP (Shapley additive explanations) framework to analyze HAR model predictions.
• Applied various validation strategies to three public HAR datasets.
• Compared feature importance across different validation methodologies.

Main Results:

• Identified that k-folds cross-validation (k-CV) can overestimate HAR prediction accuracy by approximately 13%.
• Demonstrated that k-CV selects a different feature set compared to a universal model.
• Showcased SHAP's capability to provide graphical insights into HAR model decision-making.

Conclusions:

• Explainable methods like SHAP offer a simplified way to analyze HAR system performance.
• Inappropriate validation strategies, such as k-CV, can lead to significant overestimation of accuracy and biased feature selection.
• Combining explainable AI with HAR models helps researchers understand model behavior, detect bias, and avoid performance overestimation before deployment.