Clinical Validation of Novel Digital Measures: Statistical Methods for Reliability Evaluation.

Area of Science:

• Biostatistics
• Digital Health Technology
• Clinical Measurement Validation

Background:

• Reliability assessment is a critical step in validating digital health technology (DHT) tools for clinical research and decision-making.
• It helps characterize the signal-to-noise ratio and measurement error, indicating the potential usefulness of a clinical measure.
• Current methodologies for reliability analyses are dispersed across various validation literature.

Purpose of the Study:

• To provide a comprehensive overview of statistical methodologies and analytical tools for reliability assessment of novel digital clinical measures.
• To consolidate scattered reliability analysis techniques into a generalizable framework for DHT validation.
• To equip statisticians and data scientists with the necessary tools for robust digital clinical measure validation.

Main Methods:

• Review of general modeling frameworks and statistical metrics for reliability assessment in clinical validation.
• Presentation of methods for assessing agreement and measurement error.
• Adaptation of techniques for categorical measures and illustration with wearable device accelerometer data.

Main Results:

• A unified framework for reliability assessment applicable to digital clinical measures.
• Demonstration of techniques using real-world physical activity data from a clinical trial.
• Identification of key statistical metrics and analytical tools for DHT validation.

Conclusions:

• Standardized reliability assessment is essential for the successful implementation of digital clinical measures.
• This paper serves as a valuable resource for statisticians and data scientists developing and validating DHTs.
• The presented methodologies enhance the rigor and trustworthiness of evidence generated by digital health tools.