An End-User Audit of Reproducibility, Data Leakage, and Overfitting of the Top-Ranked ADMET Prediction Models in TDC...

Public leaderboards such as the Therapeutics Data Commons (TDC) ADMET benchmark are widely treated as a ranking of state-of-the-art models. However, a high leaderboard position is only meaningful if the corresponding model can actually be reproduced and deployed by an independent researcher. In this work, we audit whether the top-ranked TDC ADMET models meet that bar. We assessed the top-ranked models of all 22 TDC ADMET leaderboards from the perspective of an end user with access only to the publicly released artifacts of each model─its publication, code repository, and installation instructions. For every end point, the top three models were screened with a unified protocol including an execution environment reproducibility check, a data-leakage assessment, verification of the hyperparameter-optimization procedure, and a reevaluation against the current leaderboard. Only three models (CaliciBoost, MapLight, and MapLight + GNN) passed all stages and reproduced their reported performance. The remaining models failed because of unavailable code, nonreproducible environments, runtime incompatibilities, or methodological flaws. We traced direct or indirect data leakage in the MiniMol, GradientBoost, and XGBoost models, and used deliberately overfitted variants of our own Mol2Vec-based models to show that tuning on the public test set─whether accidental or intentional─can substantially inflate both metrics and leaderboard rank. These results indicate that current TDC leaderboard positions cannot be read as a direct measure of model quality and practical applicability and emphasize the urgent need for better public ADMET benchmarks based on the hidden test sets, strict data set versioning and model submission with standardized inference environments.