Revisiting ADMET prediction reliability under real-world challenges in the foundation model era

Area of Science:

• Artificial Intelligence in Drug Discovery
• Machine Learning for Cheminformatics
• Computational Drug Design

Background:

• AI-driven drug discovery is shifting towards foundation models.
• Existing molecular foundation models lack systematic benchmarking for real-world challenges.
• Druggability prediction needs robust evaluation across diverse scenarios.

Purpose of the Study:

• Establish a benchmark for evaluating AI models in drug discovery.
• Assess limitations and frontiers of molecular and tabular foundation models.
• Provide guidance for selecting and improving druggability prediction models.

Main Methods:

• Developed a benchmark with four challenges: data scarcity/OOD generalization, class imbalance, beyond Rule of 5 (bRo5) generalization, and activity cliffs.
• Performed large-scale benchmarking of molecular foundation models (KPGT, Uni-Mol) and tabular foundation models (TabPFNv2).
• Evaluated ensemble strategies and the molecular property landscape roughness index.

Main Results:

• Tabular foundation models show superior generalization in few-shot and OOD settings.
• Ensemble methods with undersampling effectively handle extreme class imbalance.
• Graph Neural Networks (GNNs) like KPGT perform well in bRo5 chemical spaces with sufficient data.
• All models struggle with predicting activity cliffs, indicating a persistent challenge.

Conclusions:

• Ensembling diverse, high-performance models enhances predictive robustness.
• The molecular property landscape roughness index is a useful tool for model selection.
• Findings offer practical guidance for evaluating and selecting AI models for druggability prediction.