Using propensity score-based weighting in the evaluation of health management programme effectiveness

Area of Science:

• Health Services Research
• Epidemiology
• Biostatistics

Background:

• Randomized controlled trials (RCTs) are the gold standard for program evaluation but are often unfeasible in real-world settings.
• Observational studies attempt to emulate RCTs by creating equivalent control groups, but unknown characteristics can introduce bias.
• Propensity score methods, such as matching and subclassification, are increasingly used to adjust for observed confounders.

Purpose of the Study:

• To introduce a propensity-score weighting adjustment technique for evaluating health management programs using observational data.
• To demonstrate the implementation of propensity-score weighting using a real-world example of health coaching for chronically ill individuals.
• To highlight the utility of propensity-score weighting for unbiased treatment effect estimation in the absence of RCTs.

Main Methods:

• Utilizes propensity scores to create inverse probability of treatment weights.
• Weighting mechanism addresses imbalances in pre-intervention characteristics between treated and non-treated groups.
• Employs a regression framework with propensity-score weights to estimate treatment effects.

Main Results:

• The propensity-score weighting method effectively removes imbalances in known characteristics.
• The technique provides unbiased estimates of treatment effects in observational studies.
• Illustrates successful application in evaluating motivational interviewing-based health coaching on patient activation.

Conclusions:

• Propensity-score weighting adjustment is a powerful and relatively simple alternative for evaluating health management program effectiveness with observational data.
• This method should be considered by program evaluators seeking to mitigate bias when RCTs are not feasible.
• The technique offers a robust approach to emulate randomization and ensure control group equivalence on observed variables.