Modeling current practices in critical care comparative effectiveness research

Objective: To determine whether contemporaneous practices are adequately represented in recent critical care comparative effectiveness research studies. Design: All critical care comparative effectiveness research trials published in the New England Journal of Medicine from April 2019 to March 2020 were identified. To examine studies published in other high impact medical journals during the same period, such trials were subsequently also identified in the Journal of the American Medical Association and The Lancet. All cited sources were reviewed, and the medical literature was searched to find studies describing contemporary practices. Then, the designated control group or the comparable therapies studied were examined to determine if they represented contemporaneous critical care practices as described in the medical literature. Results: Twenty-five of 332 randomised clinical trials published in these three journals during this 1-year period described critical care comparative effectiveness research that met our inclusion criteria. Seventeen characterised current practices before enrolment (using surveys, observational studies and guidelines) and then incorporated current practices into one or more study arm. In the other eight, usual care arms appeared insufficient. Four of these trials randomly assigned patients to one of two fixed approaches at either end of a range of usually titrated care. However, due to randomisation, different subgroups within each arm received care that was inappropriate for their specific clinical conditions. In the other four of these trials, common practices influencing treatment choice were not reflected in the trial design, despite a prior effort to characterise usual care. Conclusion: One-third of critical care comparative effectiveness research trials published in widely read medical journals during a recent year did not include a designated control arm or comparable therapies representative of contemporary practices. Failure to incorporate contemporary practices into critical care comparative effectiveness trials appears to be a widespread design weakness.