Framework for evaluating explainable AI in antimicrobial drug discovery.

Area of Science:

• Computational chemistry
• Artificial intelligence in drug discovery

Background:

• Explainable AI (XAI) methods are crucial for understanding molecular property predictions in drug development.
• Lack of standardized evaluation criteria hinders the deployment of XAI in drug discovery and hit optimization.
• Understanding structure-activity relationships (SAR) is essential for medicinal chemistry.

Purpose of the Study:

• To develop and apply a novel evaluation framework for XAI methods in molecular property prediction.
• To compare the explainability of different XAI approaches using fragment-based explainability tests.
• To assess the utility of XAI for explaining activity cliffs and SAR in drug discovery.

Main Methods:

• Developed an evaluation framework incorporating scaffold recognition, sensitivity, and substructure specificity tests.
• Assessed model robustness and consistency.
• Implemented and compared three XAI models: Random Forest (SHAP), CNN (token occlusion), and RGCN (substructure masking) on antibiotic molecules.

Main Results:

• All evaluated XAI approaches demonstrated strong predictive performance and scaffold recognition.
• Models exhibited comparable robustness and consistency.
• Significant differences were observed in the explainability of activity cliffs across the XAI methods, highlighting varying utility for medicinal chemists.

Conclusions:

• The developed evaluation framework effectively differentiates the explainability behaviors of various XAI methods.
• Different XAI approaches offer distinct insights into SAR and activity cliffs, impacting their applicability in drug discovery.
• Standardized evaluation is critical for advancing XAI in medicinal chemistry and drug development.