Related Experiment Video
Updated: Mar 7, 2026
06:55
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
15.4K
Comparison of disease risk score methods to study treatment effect heterogeneity: a simulation study
Haedi E Thelen1,2, Wei Yang1, Sean Hennessy1,2
1Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, United States.
American Journal of Epidemiology
|March 6, 2026
Summary
The split-sample method for deriving disease risk scores (DRS) in randomized trials shows less bias in estimating treatment effects compared to controls-only or full-sample methods. This approach is preferable when sufficient data is available.
Area of Science:
- Biostatistics
- Clinical Trials
- Epidemiology
Background:
- Estimating treatment effect heterogeneity using disease risk scores (DRS) is crucial in randomized trials.
- The optimal internal derivation method for DRS models is uncertain when external models are unavailable.
- Assessing treatment effects across different risk strata helps understand treatment variability.
Purpose of the Study:
- To compare the bias of estimated treatment effects within DRS-defined strata using three internal DRS model derivation approaches.
- To evaluate the performance of controls-only, full-sample, and split-sample methods for internal DRS model fitting.
- To identify the most effective method for estimating treatment effect heterogeneity by DRS in randomized trials.
Main Methods:
- A simulation study was conducted comparing three internal derivation approaches for DRS models.
- Methods included fitting DRS models on controls-only, the full sample, and a random split-sample of controls.
- Simulations varied treatment effects (odds ratios), treatment-covariate interactions, outcome incidence, sample size, randomization ratio, and DRS c-statistic.
Main Results:
- The split-sample method demonstrated the lowest overall percent bias (OPB) in estimated treatment effects (7.7% for OR=0.8 with interactions).
- Controls-only (OPB=15.6%) and full-sample (OPB=22.1%) methods showed significantly higher bias.
- Bias reduction with the split-sample method was more pronounced with larger sample sizes, higher outcome incidence, greater treated-to-control ratios, and larger c-statistics.
Conclusions:
- Split-sample methods appear to be the preferred approach for estimating treatment effect heterogeneity by DRS in randomized trials.
- This method is particularly advantageous when sufficient data supports stable prediction model development.
- The findings provide guidance on selecting optimal internal DRS derivation strategies for clinical trial analysis.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
510
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
510
01:12Hazard Ratio
670
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
670
01:20Comparing the Survival Analysis of Two or More Groups
677
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
677
01:15Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
334
Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
334
01:12Relative Risk
2.3K
Relative risk (RR) is a statistical measure commonly used in epidemiology to compare the likelihood of a particular event occurring between two groups. This metric is important for evaluating the relationship between exposure to a specific risk factor and the probability of a particular outcome. It plays a crucial role in medical research, public health studies, and risk assessment. Relative risk quantifies how much more (or less) likely an event is to occur in an exposed group compared to an...
2.3K
01:20Bioequivalence Experimental Study Designs: Completely Randomized and Randomized Block Designs
347
Bioequivalence experimental study designs are crucial methodologies used in evaluating and comparing the bioavailability of different drug products. These designs are categorized into various types: completely randomized, randomized block, repeated measures, cross and carry-over, and Latin square designs.Completely randomized designs involve randomly allocating treatments to all subjects participating in the experiment. This allocation is achieved by assigning unique random numbers to subjects...
347