Ordinal labels in machine learning: a user-centered approach to improve data validity in medical settings

Area of Science:

• Medical Informatics
• Machine Learning
• Fuzzy Set Theory

Background:

• Medical acts inherently involve uncertainty, yet explicit consideration of this uncertainty in data representation is under-researched.
• Standard medical terminologies for health condition severity (e.g., pain) range from Absent to Extreme but lack precise, universally agreed-upon numerical interpretation.
• Understanding user perception of these ordinal scales is crucial for accurate data modeling.

Purpose of the Study:

• To investigate how potential patients and clinicians quantitatively and qualitatively perceive different levels of a widely used medical severity terminology.
• To develop and evaluate fuzzy-set based representations of medical ordinal scales that incorporate expert and user knowledge.
• To assess the impact of these novel encodings on the performance of machine learning models for medical prognostics.

Main Methods:

• A questionnaire-based study was conducted with 1,152 potential patients and 31 clinicians to gather numerical perceptions of standard medical severity labels.
• Fuzzy-set representations were developed based on the collected user perception data.
• The effectiveness of these representations was evaluated by applying them to common machine learning models for a real-world medical prognostic task.

Main Results:

• Significant differences were identified in the perception of pain levels between patient and clinician groups.
• The proposed fuzzy-set based ordinal encodings demonstrated an improvement in the predictive performance of specific machine learning model classes.
• The study highlights the impact of user-informed data encoding on model accuracy.

Conclusions:

• Novel techniques for ordinal scale representation and encoding can enhance the validity of medical datasets.
• Incorporating user-perceived vagueness into machine learning models can improve prognostic task performance.
• The methodology offers a framework for better understanding and utilizing ordinal medical scales in research and clinical applications.