Agreement and utility between registry and trial mortality data - a data utility comparison in the BOSS trial

Healthcare Systems Data (HSD) are increasingly used in RCTs to supplement data required for clinical trial outcomes; however, the true quality and utility of this data remain unclear. In clinical trials when data discrepancies are present, rules of data integration must be in place to handle the differences; however, there is little evidence as to the best approach for how these principles are set. The purpose of this study is to conduct a comprehensive data utility comparison of HSD mortality data and trial-specific mortality data collected in the BOSS trial.

Trial-specific and HSD mortality data collected in the BOSS trial were compared to assess levels of agreement between death status, death dates, and causes of death. Potential sources of data inconsistencies were examined to determine underlying patterns between HSD and trial-specific data discrepancies. HSD and trial-specific data were combined through five data integration approaches to assess their impact on the primary outcome of overall survival.

Death status and death dates were similar between trial-specific and HSD data (Cohen's Kappa statistic 0.866, 95% CI 0.842 to 0.889); however, more than 100 participants were recorded with inconsistent death statuses, and the median difference between different death dates was almost a year (340 days, IQR: 11 to 865 days). All data integration approaches contributed to similar treatment effect estimates, and none changed the results of the trial. The treatment effect estimations from either source exclusively were comparable (HSD Hazard Ratio 1.01, 95% CI 0.85 to 1.20; trial-specific Hazard Ratio 0.89, 95% CI 0.75 to 1.06), and when both sources were integrated to produce hazard ratio estimates, the results were almost identical regardless of the approach taken.

In a direct comparison of mortality data between HSD and trial-specific data for the BOSS trial, both sources were mostly accurate and complete. Trial results were not impacted by different approaches of data integration and were generally strengthened by combining all data sources. This should be always encouraged when trial data and HSD are both collected.

ISRCTN54190466. Registered on 8 January 2008.