A comprehensive assessment and comparison of tools for HLA class I peptide-binding prediction

Human leukocyte antigen class I (HLA-I) molecules bind intracellular peptides produced by protein hydrolysis and present them to the T cells for immune recognition and response. Prediction of peptides that bind HLA-I molecules is very important in immunotherapy. A growing number of computational predictors have been developed in recent years. We survey a comprehensive collection of 27 tools focusing on their input and output data characteristics, key aspects of the underlying predictive models and their availability. Moreover, we evaluate predictive performance for eight representative predictors. We consider a wide spectrum of relevant aspects including allele-specific analysis, influence of negative to positive data ratios and runtime. We also curate high-quality benchmark datasets based on analysis of the consistency of the data labels. Results reveal that each considered method provides accurate results, which can be explained by our analysis that finds that their predictive models capture meaningful binding motifs. Although some methods are overall more accurate than others, we find that none of them is universally superior. We provide a comprehensive comparison of the convenience as well as the accuracy of the methods under specific prediction scenarios, such as for specific alleles, metrics of predictive performance and constraints on runtime. Our systematic and broad analysis provides informative clues to the users to identify the most suitable tools for a given prediction scenario and for the developers to design future methods.