Evaluating and Enhancing the Generalization Performance of Machine Learning Models for Physical Activity Intensity
IEEE Journal of Biomedical and Health Informatics
|May 21, 2019
Summary
Machine learning models for physical activity intensity prediction generalize better when trained on diverse datasets from various activity monitors. Within-dataset validation alone is insufficient for assessing real-world performance across different devices.
Area of Science:
- Wearable technology and machine learning
- Biomedical signal processing
- Physical activity recognition
Background:
- Machine learning models are increasingly used for physical activity intensity prediction using raw acceleration data.
- Generalizability of these models across different activity monitors and populations remains a significant challenge.
- Current validation methods may not accurately reflect real-world performance.
Purpose of the Study:
- To evaluate and enhance the generalization performance of machine learning models for physical activity intensity prediction.
- To assess model performance across diverse populations and activity monitors using raw acceleration data.
- To investigate methods for improving model generalizability.
Main Methods:
- Five datasets from four studies, including hip- and wrist-based raw acceleration data, were utilized.
- Artificial neural networks (ANN) were developed and validated to classify activity intensity (sedentary, light, moderate-to-vigorous).
- Models were trained and cross-tested across different datasets and accelerometers, with strategies including leave-one-subject-out cross-validation and training on merged datasets.
Main Results:
- Within-dataset validation showed high performance (accuracy 71.9-95.4%), but performance dropped significantly when applied to different accelerometers (41.2-59.9%).
- Models trained on merged datasets (hip and wrist data) demonstrated acceptable performance on left-out datasets (65.9-83.7%).
- A single model trained on all five datasets achieved robust performance across datasets (80.4-90.7%).
Conclusions:
- Integrating heterogeneous datasets into training sets is a viable strategy for improving model generalization.
- Within-dataset validation is inadequate for assessing model performance on populations using different accelerometers.
- Developing generalized models requires training on diverse, multi-device data.
Related Concept Videos
03:00Self-Evaluation: Self-Enhancement and Self-Verification
5.8K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.8K
00:55IR Spectrum Peak Intensity: Amount of IR-Active Bonds
1.0K
When infrared radiation is passed through a molecule, absorption occurs if the molecule's vibration leads to a substantial change in its bond dipole moment. Transitions between vibrational energy levels, typically corresponding to infrared frequencies (4000–400 cm−1), allow absorption if the vibration significantly alters the dipole moment, making the molecule infrared active. The molecular bonds have different stretching and bending vibrations, resulting in various peaks with...
1.0K
01:22Generalized Hooke's Law
2.7K
The generalized Hooke's Law is a broadened version of Hooke's Law, which extends to all types of stress and in every direction. Consider an isotropic material shaped into a cube subjected to multiaxial loading. In this scenario, normal stresses are exerted along the three coordinate axes. As a result of these stresses, the cubic shape deforms into a rectangular parallelepiped. Despite this deformation, the new shape maintains equal sides, and there is a normal strain in the direction of the...
2.7K
01:30Generalized Anxiety Disorder
669
Generalized Anxiety Disorder (GAD) is a chronic condition characterized by excessive and uncontrollable worry that persists for at least six months, significantly interfering with daily functioning. Unlike situational anxiety, which arises in response to specific stressors, GAD often occurs without a clear cause. Individuals may experience disproportionate worry about work, health, or relationships. For instance, a person might continuously fear poor health despite normal medical evaluations or...
669
01:17Performing a Simple Data Analysis using MS-Excel Function
934
Microsoft Excel offers a suite of functions and tools ideal for statistical analysis, making it accessible to students and researchers. This article outlines fundamental Excel functions pivotal for data analysis.
SUM: This function calculates the total sum of a range of values. It's the foundation for aggregating data, essential for determining overall trends and totals in datasets.
AVERAGE: It computes the mean value of a given set of numbers, providing a quick insight into the central...
SUM: This function calculates the total sum of a range of values. It's the foundation for aggregating data, essential for determining overall trends and totals in datasets.
AVERAGE: It computes the mean value of a given set of numbers, providing a quick insight into the central...
934
01:30Simplified Synchronous Machine Model
759
The Synchronous Machine Model is a fundamental tool in analyzing and ensuring the transient stability of power systems. This model simplifies the representation of a synchronous machine under balanced three-phase positive-sequence conditions, assuming constant excitation and ignoring losses and saturation. The model is pivotal for understanding the behavior of synchronous generators connected to a power grid, particularly during transient events.
In this model, each generator is connected to a...
In this model, each generator is connected to a...
759