Evaluating the Performance of Sensor-based Bout Detection Algorithms: The Transition Pairing Method.

Area of Science:

• Wearable sensor technology
• Biomedical engineering
• Data analysis algorithms

Background:

• Bout detection algorithms segment data from wearable sensors.
• Assessing the correctness of bout detection segmentation is challenging.
• Accurate segmentation is crucial for analyzing physical activity and behavior patterns.

Purpose of the Study:

• To introduce and demonstrate the Transition Pairing Method (TPM).
• To provide a novel method for evaluating bout detection algorithm performance.
• To standardize the assessment of segmentation correctness in wearable sensor data.

Main Methods:

• The TPM compares predicted transitions to a criterion measure based on number and timing.
• A true positive is a predicted transition matched to a criterion transition within a defined time lag.
• The Gale-Shapley algorithm pairs transitions; unpaired predictions/criteria are false positives/negatives.

Main Results:

• With a 1-s lag, recall was <20% and precision was <10% for tested algorithms.
• Over 80% of criterion transitions were missed, and over 90% of predicted transitions were incorrect.
• Significant differences were observed between algorithm performance metrics.

Conclusions:

• The TPM offers a standardized approach to evaluating bout detection algorithms.
• It provides specific, actionable insights into algorithm performance beyond conventional metrics.
• This method enhances the reliability of wearable sensor data analysis for research.