Assessing Representativeness of Randomised Controlled Trials using Serious Adverse Events

Context: The applicability of randomised controlled trials of pharmacological agents to older people with frailty/multimorbidity is often uncertain, due to concerns that trials are not representative. However, assessing trial representativeness is challenging and complex. Objectives: We explore an approach assessing trial representativeness by comparing rates of trial Serious Adverse Events (SAEs: most of which reflect hospitalisations/deaths) to rates of hospitalisation/death in routine care (which, in a trial setting, would be SAEs be definition). Study design: Secondary analysis of trial and routine healthcare data. Dataset and population: 483 trials (n=636,267) from clinicaltrials.gov across 21 index conditions. A routine care comparison was identified from SAIL databank (n=2.3M). Instrument: SAIL data were used to derive the expected rate of hospitalisations/deaths by age, sex and index condition. Outcomes: We calculated the expected number of SAEs for each trial compared to the observed number of SAEs (observed/expected SAE ratio). We then re-calculated the observed/expected SAE ratio additionally accounting for comorbidity count in 125 trials for which we accessed individual participant data. Results: For 12/21 index conditions the observed/expected SAE ratio was <1, indicating fewer SAEs in trials than expected given community rates of hospitalisations and deaths. A further 6/21 had point estimates <1 but the 95% CI included the null. The median observed/expected SAE ratio was 0.60 (95% CI 0.56-0.65; COPD) and the interquartile range was 0.44 (0.34-0.55; Parkinson's disease) to 0.88 (0.59-1.33; IBD). Higher comorbidity count was associated with SAEs/hospitalisations and deaths across index conditions. For most trials, the observed/expected ratio was attenuated but remained <1 after additionally accounting for comorbidity count. Conclusion: Trial participants experience fewer SAEs than expected based on age/sex/condition hospitalisation and death rates in routine care, confirming the predicted lack of representativeness. This difference is only partially explained by differences in multimorbidity. Assessing observed/expected SAE may help assess applicability of trial findings to older populations in whom multimorbidity and frailty are common.