The impact of inconsistent human annotations on AI driven clinical decision making

Area of Science:

• Machine Learning
• Clinical Informatics
• Medical AI

Background:

• Supervised learning in healthcare relies on expert annotations, but inconsistencies arise from expert bias and judgment variations.
• The impact of these annotation inconsistencies on real-world clinical model performance is understudied.

Purpose of the Study:

• To investigate the implications of expert annotation inconsistencies in supervised learning for Intensive Care Unit (ICU) datasets.
• To evaluate current best practices for establishing gold-standard models and consensus in clinical AI development.

Main Methods:

• Developed individual supervised learning models using a common ICU dataset annotated independently by 11 clinical experts.
• Performed internal validation using Fleiss' kappa and external validation on a HiRID dataset using Cohen's kappa.
• Analyzed model agreement for discharge decisions versus mortality prediction.

Main Results:

• Internal validation showed fair agreement (Fleiss' κ = 0.383) among expert models.
• External validation revealed low pairwise agreement (average Cohen's κ = 0.255) between models.
• Models showed less agreement on discharge decisions (Fleiss' κ = 0.174) than mortality prediction (Fleiss' κ = 0.267).
• Standard consensus methods like majority voting produced suboptimal models.

Conclusions:

• No single 'super expert' reliably emerged, and simple consensus approaches are insufficient for optimal clinical AI.
• Assessing annotation learnability and using 'learnable' subsets for consensus building can lead to optimal models.