Cross-sectional analysis of accuracy versus interpretability in Medicare Advantage risk adjustment

Risk adjustment models in Medicare Advantage determine annual payments of over $300 billion in public funds to private companies. Policymakers want risk adjustment models that are both accurate and interpretable to ensure appropriate use of public funds.

The trade-off between accuracy and interpretability from using standard machine learning (ML) models for risk adjustment was evaluated. A cross-sectional analysis was conducted using 2018-2019 Medicare claims with 3,602,618 beneficiaries. Multiple risk adjustment models were estimated, including traditional and ML-based approaches. Performance was assessed using out-of-sample mean absolute and squared error (MAE and MSE). Interpretability was measured using coefficient count and log-transformed coefficient count of models.

ML models, especially gradient-boosted trees, significantly improved prediction accuracy relative to recent Medicare models, with MAE reductions of - 1,352 (95 % CI: -1,392, -1,316) and MSE reductions of - 5 (95 % CI: -9, -1). However, these improvements increased model complexity by more than 1000x and provided less than 0.1 % of the accuracy improvement per additional coefficient of a previous major model change. Notably, the predictions from gradient-boosted trees responded less to strategic diagnosis coding, reducing incentives to upcode.

Standard ML models can modestly improve predictive accuracy but substantially worsen model interpretability in risk adjustment. Future research is needed to improve accuracy in these models while maintaining the interpretability essential for oversight of public spending.