Bridging generalizability and reproducibility in machine learning-based antimicrobial resistance prediction

The rapid growth of whole-genome sequencing and machine learning (ML) has led to a multitude of methods for predicting antimicrobial resistance (AMR) from bacterial genomic data. Despite increasingly high reported predictive performance, substantial differences in data sets, phenotype definitions, genomic preprocessing, evaluation strategies, and validation procedures make it difficult to determine whether models will perform reliably beyond the data on which they were developed. In machine learning, this problem is generally framed in terms of generalizability: the ability of a model to maintain performance when applied to previously unseen data. In microbiology, many of the same considerations ultimately manifest as a problem of reproducibility. In this review, we provide a brief overview of the fundamental concepts underlying machine learning-based AMR prediction and then examine the factors that affect model generalizability and reproducibility.