QMAP: a benchmark for standardized evaluation of antimicrobial peptide MIC and hemolytic activity regression

Area of Science:

• Computational chemistry
• Biotechnology
• Drug discovery

Background:

• Antimicrobial peptides (AMPs) show promise as alternatives to conventional antibiotics.
• Computational AMP discovery lacks standardized benchmarks, hindering progress assessment.
• Inconsistent datasets and evaluation protocols impede reliable quantification of model performance.

Purpose of the Study:

• To introduce QMAP, a domain-specific benchmark for evaluating AMP discovery models.
• To standardize the assessment of antimicrobial potency (MIC) and hemolytic toxicity (HC50) prediction.
• To enforce homology-aware evaluation for true model generalization.

Main Methods:

• Development of QMAP, a benchmark with homology-aware, predefined test sets.
• Application of QMAP to reassess existing MIC prediction models.
• Establishment of baseline performance metrics for MIC and HC50 regression.

Main Results:

• Limited progress in AMP discovery model performance over the past six years.
• Poor performance observed for high-potency MIC regression tasks.
• Low predictability for hemolytic activity (HC50) across evaluated models.
• QMAP facilitates standardized evaluation and highlights areas for methodological improvement.

Conclusions:

• Standardized evaluation is crucial for advancing computational AMP discovery.
• Current models show limitations in predicting high-potency antimicrobial activity and hemolytic toxicity.
• Improved modeling approaches are needed for developing effective and safe AMPs.
• The QMAP benchmark and associated Python package (pip install qmap-benchmark) promote practical adoption and further research.