Evaluating NetMHCpan performance on non-European HLA alleles not present in training data

Area of Science:

• Immunoinformatics
• Computational Biology
• Machine Learning in Healthcare

Background:

• Bias in neural network training datasets can reduce prediction accuracy for underrepresented groups, impacting healthcare applications.
• Machine learning models like NetMHCpan-4.1 and NetMHCIIpan-4.0 predict antigen binding to MHC molecules, critical for adaptive immune responses.
• Previous studies indicated potential bias in these models towards hydrophobic peptides.

Purpose of the Study:

• To investigate the composition of training datasets for NetMHCpan-4.1 and NetMHCIIpan-4.0, focusing on ethnic representation.
• To evaluate the prediction accuracy of these models for alleles not included in their training datasets.
• To understand the relationship between training data disparities, HLA sequence diversity, and prediction performance.

Main Methods:

• Analysis of training dataset composition for ethnic bias (European Caucasian, Asian, Pacific Islander).
• Testing NetMHCpan-4.1 and NetMHCIIpan-4.0 performance on alleles absent from training data.
• Mapping HLA sequence space to assess training dataset diversity.
• Linking impactful residues in NetMHCpan predictions to structural features for specific alleles.

Main Results:

• The training datasets for NetMHCpan-4.1 and NetMHCIIpan-4.0 were found to be heavily biased against Asian and Pacific Islander individuals, favoring European Caucasians.
• Unexpectedly, no meaningful difference in prediction quality was observed for alleles not present in the training data, despite the identified ethnic disparities.
• Analysis revealed the sequence diversity of the training dataset and linked key prediction residues to structural features.

Conclusions:

• Ethnic bias in training data for MHC-binding prediction models does not necessarily translate to decreased prediction quality for underrepresented alleles.
• Further investigation into HLA sequence space and structural features is necessary to fully understand model behavior and potential biases.
• Ensuring equitable performance of machine learning models in healthcare requires careful consideration of training data composition and its impact on diverse populations.