[Control group formation using propensity score matching: The role of primary and secondary data - Results of...

The creation of control groups in the evaluation of statutory health insurances is a key issue. Randomization represents both an ethical and a legal problem with legally guaranteed services. Matching procedures are relevant alternatives in the construction of control groups. Matchings are mostly based on secondary data from statutory health insurances (for example age, gender, cost of illness, days of incapacity to work). In this study, we examined whether matching based on secondary data alone can cause selection bias.

We used data from three large prevention studies and applied sensitivity analyses to compare the results of propensity score matchings used to create control groups on the basis of secondary data, with those obtained on the basis of both primary and secondary data. Analysis of covariance was used to investigate the impact of potential selection bias on cost effects.

Matchings based on secondary data alone lead to control groups with similar characteristics captured by secondary data. However, the control group participants are significantly healthier (they have, for example, lower levels of pain, lower levels of psychological stress, a higher degree of quality of life) than the patients in intervention groups. This selection bias would lead to a systematic underestimation of the cost reduction produced by preventive interventions.

Prevention course participants seem to have characteristics that differ from the average population (higher health orientation level, preference for prevention over medical treatment services, etc.) and cannot be captured by secondary data; therefore, matchings based on secondary data alone cause selection bias.

Including both primary and secondary data reduces the risk of selection bias in matching procedures for prevention studies. The E-value can be used to evaluate the robustness of results with regard to selection bias.