A data-driven framework for identifying patient subgroups on which an AI/machine learning model may underperform.

Area of Science:

• Clinical informatics
• Machine learning evaluation
• Health data science

Background:

• Evaluating clinical models across diverse patient populations is crucial.
• Heterogeneous patient subgroups in development data can mask performance disparities.
• Average model performance may hide significantly lower performance in specific subgroups.

Purpose of the Study:

• To introduce an algorithmic framework for identifying subgroups with potential performance disparities (AFISP).
• To enable detection of interpretable phenotypes linked to lower model performance.
• To facilitate identification of potential clinical model failure modes prior to deployment.

Main Methods:

• Development of an algorithmic framework for identifying subgroups with potential performance disparities (AFISP).
• Generation of interpretable phenotypes corresponding to subgroups with lower model performance.
• Application of AFISP to a patient deterioration model.

Main Results:

• AFISP successfully identified significant subgroup performance disparities.
• The framework produces interpretable phenotypes for targeted evaluation.
• AFISP demonstrated significantly greater scalability compared to existing methods.

Conclusions:

• AFISP provides a scalable and interpretable method for detecting performance disparities in clinical models.
• This framework aids in identifying potential failure modes in specific patient subgroups.
• AFISP enhances the robustness of clinical model evaluation across diverse populations.