Proposal of a New Rating Concept for Digital Health Applications in Orthopedics and Traumatology

Health-related mobile applications (apps) are rapidly increasing in number. There is an urgent need for assessment tools and algorithms that allow the usability and content criteria of these applications to be objectively assessed. The aim of this work was to establish and validate a concept for orthopedic societies to rate health apps to set a quality standard for their safe use.

An objective rating concept was created, consisting of nine quality criteria. A self-declaration sheet for app manufacturers was designed. Manufacturers completed the self-declaration, and the app was examined by independent internal reviewers. The pilot validation and analysis were performed on two independent health applications. An algorithm for orthopedic societies was created based on the experiences in this study flow.

"Sprunggelenks-App" was approved by the reviewers with 45 (98%) fulfilled criteria and one (2%) unfulfilled criterion. "Therapie-App" was approved, with 28 (61%) met criteria, 6 (13%) unfulfilled criteria and 12 (26%) criteria that could not be assessed. The self-declaration completed by the app manufacturer is recommended, followed by a legal and technical rating performed by an external institution. When rated positive, the societies' internal review using independent raters can be performed. In case of a positive rating, a visual certification can be granted to the manufacturer for a certain time frame.

An objective rating algorithm is proposed for the assessment of digital health applications. This can help societies to improve the quality assessment, quality assurance and patient safety of those apps. The proposed concept must be further validated for inter-rater consistency and reliability.