Quantifying how small variations in design elements affect risk in an incident cohort study in claims

Area of Science:

• Pharmacoepidemiology
• Biostatistics
• Health Services Research

Background:

• Epidemiological study reporting lacks transparency for evaluation and replication.
• Insufficient detail on study design elements is a significant barrier.

Purpose of the Study:

• Investigate the impact of key design element variations on risk estimation.
• Utilize a drug safety evaluation in claims data as a test case.

Main Methods:

• Employed a fractional factorial design to covary five key elements: index day, exposure duration algorithms, heparin exclusion, propensity score variables, and Cox model stratification.
• Generated 24 risk estimates for one outcome and repeated eight combinations for two additional outcomes.
• Measured effects on cohort size, follow-up time, and risk estimates.

Main Results:

• Index day and exposure algorithms had the most substantial impact on risk estimation.
• These elements altered cohort size by 8-10%, follow-up time by up to 31%, and log hazard ratios by up to 0.22.
• Design changes significantly altered matched control group composition.

Conclusions:

• Exposure-related design elements are critical for study investigators and analysts.
• Developed methods using factorial design are applicable for planning sensitivity analyses in similar epidemiological studies.