Time-split cross-validation as a method for estimating the goodness of prospective prediction

Area of Science:

• Quantitative structure-activity relationship (QSAR) modeling
• Computational chemistry
• Drug discovery

Background:

• Cross-validation is essential for validating QSAR models, assessing their self-consistency and predictive ability.
• Traditional cross-validation methods like random selection can yield overly optimistic predictions.
• Other methods, such as leave-class-out, may be too pessimistic.

Purpose of the Study:

• To evaluate different cross-validation strategies for QSAR model validation.
• To determine which cross-validation method best reflects true prospective prediction accuracy.
• To propose an improved standard for QSAR model cross-validation.

Main Methods:

• Comparison of R-squared (R²) values obtained from different test set selection strategies.
• Implementation of random selection, leave-class-out analog, and time-split selection.
• Analysis of how test set composition influences the estimation of model predictivity.

Main Results:

• Time-split selection yields R² values that more closely align with true prospective prediction.
• Random selection consistently overestimates model predictivity.
• The analog of leave-class-out selection tends to underestimate model predictivity.

Conclusions:

• Time-split selection offers a more reliable assessment of QSAR model predictive performance.
• It is recommended to incorporate time-split selection alongside random selection in QSAR model building.
• This approach enhances the robustness and trustworthiness of QSAR model validation.