Using big data to retrospectively validate the COMPASS-CAT risk assessment model: considerations on methodology

External validation is a prerequisite in order for a prediction model to be introduced into clinical practice. Nonetheless, methodologically intact external validation studies are a scarce finding. Utilization of big datasets can help overcome several causes of methodological failure. However, transparent reporting is needed to standardize the methods, assess the risk of bias and synthesize multiple validation studies in order to infer model generalizability. We describe the methodological challenges faced when using multiple big datasets to perform the first retrospective external validation study of the Prospective Comparison of Methods for thromboembolic risk assessment with clinical Perceptions and AwareneSS in real life patients-Cancer Associated Thrombosis (COMPASS-CAT) Risk Assessment Model for predicting venous thromboembolism in patients with cancer. The challenges included choosing the starting point, defining time sensitive variables that serve both as risk factors and outcome variables and using non-research oriented databases to form validated definitions from administrative codes. We also present the structured plan we used so as to overcome those obstacles and reduce bias with the target of producing an external validation study that successfully complies with prediction model reporting guidelines.