A computational validation for the health concept maturity levels questionnaire.

Area of Science:

• Healthcare Innovation
• Health Informatics
• Psychometrics

Background:

• The healthcare market demands user-centered design integration.
• Living Labs facilitate co-creation and evaluation of health innovations.
• Assessing health concept maturity is crucial for product and service development.

Purpose of the Study:

• To develop and validate a Health Concept Maturity Levels (CMLH) questionnaire.
• To introduce computational semantic validity using machine learning for health concepts.
• To evaluate the CMLH questionnaire's psychometric properties.

Main Methods:

• Developed the 178-item CMLH questionnaire.
• Annotated expert speech acts related to Health Concept Maturity Levels.
• Applied CatBoost and neural networks for computational semantic validation.
• Assessed model performance in identifying CMLH factors from text.

Main Results:

• Models trained with true factors outperformed those with random factors in identifying CMLH criteria.
• Machine learning models demonstrated ability to discern factors from speech acts.
• The CMLH questionnaire showed evidence of convergent and content validity.
• Overlaps between 'Programmatic' and 'Need' factors were identified.

Conclusions:

• Computational semantic validity offers a novel approach to psychometric validation in health innovation.
• The CMLH questionnaire is a validated tool for assessing health concept maturity.
• Further refinement of the CMLH model is needed to address factor overlaps.
• The computational method shows potential for validating other psychometric tools.