Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from...

Area of Science:

• Computational Chemistry
• Machine Learning
• Drug Discovery

Background:

• Graph neural networks (GNNs) are increasingly used for molecular property prediction.
• Existing benchmarks for GNNs often use limited traditional machine learning approaches and datasets.
• Realistic benchmarking requires diverse molecular features and external datasets for industrial applications.

Purpose of the Study:

• To benchmark various GNN models against traditional machine learning methods for molecular property prediction.
• To evaluate GNN performance using realistic benchmarks including molecular descriptors and external data.
• To identify the most promising GNN architecture for drug discovery applications.

Main Methods:

• Benchmarking of four GNN variants: Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), and Attentive Fingerprint (AttentiveFP).
• Comparison against traditional machine learning models using engineered molecular features (fingerprints, descriptors).
• Utilized time-split test sets from Genentech data and an external chemical space from Roche data.

Main Results:

• All evaluated deep learning models significantly outperformed lower-bar traditional models based solely on fingerprints.
• Graph Attention Network (GAT) demonstrated a small but consistent improvement over higher-bar traditional models.
• The accuracy of GAT single-task models was comparable to the variability observed in in vitro assays.

Conclusions:

• Graph Attention Networks (GAT) represent a promising deep learning approach for molecular property prediction.
• GAT models offer a competitive alternative to traditional machine learning methods, especially with comprehensive feature sets.
• The study suggests that experimental error propagation is a significant factor limiting model accuracy in ADME datasets.