Internal validation protocol for large collaborative clinical data sets: assessment of the CONGRESS database

Multicentre clinical research collaboratives collect large, generalisable data sets. However, data are often collected by trainees who may lack clinical or academic experience, raising concerns about data quality and potential reporting bias. Validation practices in such studies are variable. This study outlines the methods, feasibility, and outcomes of internal data validation using the CONGRESS database.

The multicentre CONGRESS data set of early oesophagogastric cancer was assessed. A random 20% sample of patients was selected to meet a >15% target validation size. Patient, disease and outcome data were re-abstracted from medical records and entered into a validation data set, which was compared with the original database. Cohen's kappa coefficient (κ) and Pearsons corelation (r) were calculated to express the strength of agreement between categorical and continuous variables, respectively.

In total, 302 patients (18.1%) from the original CONGRESS database were included in the validation data set and 3,320 data points were compared between data sets (6,640 total). The percentage of exact agreement for variables ranged from 82.5% to 98.7% (median 92.3%, interquartile range 86.3%-95.7%). Nine variables (1,645 of 2,946, 55.8% data points) showed 'almost perfect' agreement (κ or r > 0.8), and five (1,301 of 2,946, 44.2%) showed substantial agreement (κ > 0.6). None showed weak or poor agreement.

This study proposes a reproducible framework and benchmarks for validating large collaborative clinical data sets, using the national CONGRESS data set as an example. This approach offers a standard for ensuring reliable, high-quality research outcomes across multicentre databases.