Comment on "Using genomic data and machine learning to predict antibiotic resistance: A tutorial paper"

Area of Science:

• Bioinformatics
• Machine Learning
• Genomics

Background:

• A recent study proposed a machine learning curriculum for predicting antibiotic resistance using genomics data.
• The curriculum offers a step-by-step guide to traditional machine learning pipelines, accessible to novices.

Purpose of the Study:

• To critique the performance evaluation methods suggested in the aforementioned study.
• To advocate for the use of Matthews correlation coefficient (MCC) over accuracy and F1 score for binary classification.
• To highlight the potential pitfalls of k-fold cross-validation in machine learning model development.

Main Methods:

• Formal comment and critique of a published teaching curriculum.
• Explanation of statistical metrics for binary classification performance.
• Discussion of data partitioning techniques in machine learning.

Main Results:

• Accuracy and F1 score are deemed misleading metrics for binary classification tasks.
• Matthews correlation coefficient (MCC) is proposed as a more robust and reliable metric.
• K-fold cross-validation is identified as a method with significant flaws and potential pitfalls.

Conclusions:

• The Matthews correlation coefficient (MCC) should be prioritized for evaluating binary classification models in genomics.
• Caution is advised regarding the application of k-fold cross-validation due to its inherent limitations.
• Accurate model evaluation is crucial for reliable predictions in antibiotic resistance research.