Weak correlations in health services research: Weak relationships or common error?

Area of Science:

• Health Services Research
• Biostatistics
• Health Economics

Background:

• Estimating provider-specific effects on patient outcomes is crucial for quality assessment.
• Separate estimation of provider effects for different patient populations can lead to biased correlations.
• The impact of estimation methodology on correlation accuracy is not well understood, particularly with varying sample sizes.

Purpose of the Study:

• To investigate if jointly modeling provider random effects underestimates the correlation between provider effects on different patient populations.
• To compare joint modeling with stratified estimation for accuracy and bias.
• To determine how sample size influences the accuracy of these estimation procedures.

Main Methods:

• A three-pronged approach combining analytical derivation, simulation experiments, and administrative data analysis.
• Utilized traditional Medicare fee-for-service claims data for emergency department (ED) visits from January 2012 to September 2015.
• Compared results from joint modeling of random effects against stratified analyses.

Main Results:

• Joint modeling yielded estimates close to unbiased, while stratified approaches showed significant bias in small samples due to bivariate shrinkage benefits.
• In administrative data, joint modeling estimated doctor admission tendency correlations between patient groups (e.g., female/male) at 0.98, versus 0.38 for stratified estimation.
• Similar discrepancies were observed for White/non-White (0.99 vs. 0.28) and dual-eligible/non-dual-eligible (0.99 vs. 0.31) patient groups, aligning with analytical derivations.

Conclusions:

• Joint modeling accurately targets the primary parameter of interest, providing substantially less biased and higher magnitude estimates of population correlations.
• Stratified models, when post-processed, yield naive estimators that underestimate true correlations.
• The findings underscore the superiority of joint modeling for robust estimation of provider effects across diverse patient populations.