Benchmarking Molecular Feature Attribution Methods with Activity Cliffs

Area of Science:

• Computational Chemistry and Cheminformatics
• Explainable Artificial Intelligence (XAI)
• Drug Discovery and Medicinal Chemistry

Background:

• Feature attribution techniques in explainable artificial intelligence (XAI) help identify relevant input features for supervised learning models.
• In molecular design, these methods often involve coloring molecular graphs to guide medicinal chemists in prioritizing compounds for synthesis.
• Quantitative evaluation of molecular coloring approaches has been limited, primarily focusing on substructure identification.

Purpose of the Study:

• To develop and present a novel benchmark for quantitatively evaluating molecular feature attribution techniques in drug design.
• To compare the performance of classical machine learning models against graph neural network alternatives using this benchmark.
• To facilitate the development and testing of new molecular feature attribution methods by providing open-source benchmark data.

Main Methods:

• Developed a benchmark based on maximum common substructure algorithms applied to experimentally determined activity cliffs.
• Applied molecule coloring approaches using both classical machine learning models and graph neural networks.
• Quantitatively evaluated the performance of these attribution methods on the activity cliff benchmark.

Main Results:

• The proposed benchmark demonstrates that molecule coloring approaches combined with classical machine learning models generally outperform more recent graph neural network alternatives.
• Activity cliffs, crucial for understanding structure-activity relationships, serve as an effective basis for evaluating feature attribution methods.
• The study provides a valuable, open-source dataset for the advancement of molecular feature attribution techniques.

Conclusions:

• Classical machine learning models show strong performance in molecular feature attribution tasks when evaluated on activity cliffs, challenging the dominance of newer deep learning methods.
• The developed benchmark provides a standardized and quantitative method for assessing the reliability of feature attribution techniques in drug design.
• Open-sourcing the benchmark data will accelerate research and development in explainable AI for molecular discovery.