An Approach to Identifying and Quantifying Bias in Biomedical Data.

Area of Science:

• Machine Learning
• Biomedical Data Analysis
• Statistical Modeling

Background:

• Data biases impede trustworthy machine learning in biomedical fields.
• Unrepresentative labeled data requires methods utilizing unlabeled data.
• Identifying and quantifying bias is crucial for reliable AI models.

Purpose of the Study:

• To develop methods for detecting and quantifying bias in labeled data within a semi-supervised learning framework.
• To address the challenge of unrepresentative data in machine learning for biomedical applications.

Main Methods:

• Utilized a binary semi-supervised setting assuming component distributions with varying proportions in labeled and unlabeled data.
• Modeled training data using nested mixtures of multivariate Gaussian distributions.
• Developed a multi-sample expectation-maximization algorithm to learn model parameters.
• Created a statistical test to detect bias and estimated bias level using distribution distances.

Main Results:

• Successfully applied a novel expectation-maximization algorithm to learn model parameters from combined labeled and unlabeled data.
• Developed and validated a statistical test for detecting general bias in labeled data.
• Quantified the extent of bias by measuring the distance between class-conditional distributions.
• Demonstrated the effectiveness of the bias estimation procedure on both synthetic and real-world biomedical data.

Conclusions:

• The developed methods can effectively identify and quantify bias in labeled data for machine learning.
• This approach is crucial for building trustworthy AI in biomedical applications by mitigating the impact of data representativeness issues.
• The bias estimation procedure is shown to be both feasible and effective in practice.