Learning medical diagnosis models from multiple experts.

Area of Science:

• Machine Learning
• Medical Informatics
• Probabilistic Modeling

Background:

• Clinical data classification models require expert labeling, which is challenging due to expert disagreement on complex medical data.
• Learning from multiple annotators is an emerging solution to address label ambiguity and improve model robustness.

Purpose of the Study:

• To develop a novel probabilistic approach for building classification models using labels from multiple experts.
• To explicitly model annotator characteristics including prediction model, consistency, and bias.

Main Methods:

• A probabilistic framework was developed to integrate multiple expert labels.
• The method incorporates annotator-specific prediction models, consistency, and bias into the learning process.
• The approach was applied to physician-labeled clinical records for Heparin Induced Thrombocytopenia (HIT).

Main Results:

• The proposed method yields a superior classification model compared to traditional approaches.
• The framework successfully models individual annotator behavior and characteristics.
• The study demonstrates effective consensus model generation from diverse expert opinions.

Conclusions:

• This probabilistic approach enhances clinical classification by leveraging multiple expert labels.
• The method provides insights into annotator behavior, aiding in understanding label variations.
• The developed technique offers a robust solution for building reliable models from complex, multi-annotator clinical data.