Do the observational studies using propensity score analysis agree with randomized controlled trials in the area of...

Sepsis is a leading cause of mortality and morbidity in the intensive care unit, and many studies have been conducted aiming to improve its outcome. Randomized controlled trials (RCTs) and observational studies using propensity score (PS) method are commonly used for this purpose. However, the agreement between these two major methodological designs has never been investigated in this specific area. The present study aimed to compare the effect sizes between RCTs and PS-based studies.

Electronic databases including Pubmed, Scopus, and EBSCO were searched to obtain PS-based studies in the area of sepsis. The studies were matched to RCTs or systematic reviews and meta-analysis in terms of population, intervention, control, and outcome. When there were multiple PS-based studies or RCTs in one area, the effect sizes were pooled by using random-effects model and inverse variance method. The comparisons were performed by using differences in the effect size.

A total of 8 topics were identified fulfilling the criterion that at least 1 pair of RCT and PS-based study could be matched. The interventions included activated protein C, low-dose steroid, antithrombin III, combination antibiotic therapy, fish oil supplementation, statin, etomidate for intubation, and recombinant human soluble thrombomodulin. The effect sizes were statistically different between RCTs and PS-based studies in most circumstances (6/8). The pooled mean difference in effect sizes was -0.16 (95% confidence interval, -0.33 to 0.01), indicating a trend towards larger treatment effect in PS studies than in RCTs. The result remains unaltered by restricting to RCTs and PS studies with the largest sample sizes.

Our study shows that PS studies tend to report larger treatment effect than RCTs in the field of sepsis, indicating the difference between efficacy trials and effectiveness studies.