Equitable Health Intelligence: An Open Benchmark of Multi-Population Machine Learning for Omics-Based Cancer...

Area of Science:

• Biomedical informatics
• Genomic medicine
• Artificial intelligence in oncology

Background:

• Machine learning (ML) models for omics-based cancer prognosis often exhibit performance disparities across different ancestral populations due to training data biases.
• Existing fairness benchmarks primarily focus on outcome parity, neglecting predictive performance parity, which is crucial for equitable healthcare.
• There is a critical need for public benchmark resources to systematically identify and mitigate performance disparities in multi-population cancer prognosis ML models.

Purpose of the Study:

• To develop an open-source benchmark, Equitable Health Intelligence (EHI), for evaluating multi-population machine learning models in omics-based cancer prognosis.
• To provide a platform for systematically detecting and addressing performance disparities between majority and data-disadvantaged populations in cancer prognosis models.
• To facilitate the development of equitable AI solutions in precision oncology by addressing biomedical data inequality.

Main Methods:

• Developed Equitable Health Intelligence (EHI), an open-source benchmark comprising 1,475 ML tasks across 40 cancer types, 4 omics feature sets, and 3 data-disadvantaged population groups.
• Trained deep neural network models using three multi-population ML schemes (Mixture, Independent, Transfer Learning) and a Naive Transfer control, totaling 10,325 ML experiments.
• Included interactive visualization and exploratory tools for inspecting performance disparities and evaluating mitigation strategies.

Main Results:

• The EHI platform enables users to inspect predictive performance disparities between European-ancestry and data-disadvantaged populations.
• Users can evaluate the effectiveness of transfer learning in mitigating performance disparities across various cancer types, omics features, and clinical endpoints.
• The benchmark facilitates the examination of feature engineering impacts on model performance across diverse populations.

Conclusions:

• EHI serves as an open, interactive, and extensible benchmark for identifying and addressing performance disparities in multi-population ML for omics-based cancer prognosis.
• It provides a foundation for developing methods to mitigate ML performance disparities stemming from data inequality and population shifts.
• EHI advances equitable AI in precision oncology by promoting fairness and improving predictive accuracy for all populations.