Related Experiment Video
Updated: May 23, 2025
08:40
Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
Published on: June 12, 2019
7.4K
Testing and Iterative Improvement of the CEN ISO/TS 82304-2 Health App Quality Assessment: Pilot Interrater
Anna-Lena Frey1, Diana Matei1, Ben Phillips1
1Organisation for the Review of Care and Health Apps (ORCHA), Daresbury, United Kingdom.
JMIR Formative Research
|March 10, 2025
Summary
This study tested the CEN ISO/TS 82304-2 health app assessment framework across international organizations. Findings show promising interrater reliability and clarity, supporting its development for consistent global health app evaluation.
Area of Science:
- Digital Health
- Health Informatics
- Software Engineering
Background:
- Growing use of health apps necessitates robust quality assessment frameworks.
- Existing frameworks face scalability issues and lack international standardization.
- The CEN ISO/TS 82304-2 initiative aims for a unified, global health app evaluation.
Purpose of the Study:
- To refine and validate the draft CEN ISO/TS 82304-2 assessment handbook and developer guidance.
- To test the framework's applicability across diverse international organizations.
Main Methods:
- Engaged 5 international assessment organizations to evaluate 24 health apps over 3 rounds.
- Assessed interrater reliability between organizations and collected developer/assessor feedback on time and clarity.
- Iteratively improved the handbook and guidance based on collected data.
Main Results:
- Interrater reliability improved across evaluation rounds, with 80% agreement on assessment questions in round 3.
- Median preparation time for developers was 8 hours 59 minutes; median evaluation time for assessors was 8 hours 46 minutes.
- The draft guidance and handbook were rated as clear by both developers and assessors.
Conclusions:
- This study is the first to assess cross-organizational consistency in health app evaluations internationally.
- Promising early results for interrater reliability and clarity support the ongoing development of CEN ISO/TS 82304-2.
- The framework has the potential to reduce duplicated work, promote market access, and enhance user trust in health apps.
Related Concept Videos
01:29Reliability and Validity
12.6K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.6K
01:23Assessment of blood pressure in brachial artery(two-step method)
644
Measuring blood pressure is a fundamental skill in healthcare that aids in diagnosing and monitoring hypertension and other cardiovascular conditions. An aneroid sphygmomanometer, commonly used in clinical settings, offers a manual and precise method for blood pressure measurement. The technique for using this instrument involves specific steps that must be carefully executed to ensure accuracy. The following detailed description outlines a two-step technique for assessing blood pressure using...
644