Graphical displays for assessing covariate balance in matching studies

Area of Science:

• Epidemiology
• Biostatistics
• Health Services Research

Background:

• Ensuring comparability of study groups on pre-intervention characteristics is crucial for valid matching study outcomes.
• Numerical diagnostics like t-tests assess covariate balance but only capture single-dimensional equality (e.g., means).
• These tests are sensitive to sample size, potentially misinterpreting reduced power as improved balance.

Purpose of the Study:

• To demonstrate the limitations of numerical diagnostics in assessing covariate balance.
• To highlight the advantages of visual displays for a comprehensive balance assessment.
• To advocate for the use of graphical methods in reporting matching study results.

Main Methods:

• Generation of artificial datasets to illustrate diagnostic shortcomings.
• Application of widely used equality tests to these datasets.
• Utilization of various graphical displays to visualize covariate distributions.

Main Results:

• Equality tests indicated balance in means but failed to detect imbalances in variances.
• Graphical displays readily identified discrepancies missed by numerical tests.
• Smaller sample sizes created an illusion of covariate balance due to lower statistical power.

Conclusions:

• Numerical diagnostics have inherent limitations for robust covariate balance assessment.
• Graphical displays provide a more complete and reliable method for evaluating balance.
• Investigators should prioritize and include graphical displays when reporting balance statistics in matching studies.