Implementation Variation in Natural Experiments of State Health Policy Initiatives.

Area of Science:

• Health policy research
• Public health implementation science
• Healthcare systems innovation

Background:

• Federal initiatives offer states flexibility, leading to varied implementation strategies and outcomes.
• The Centers for Medicare & Medicaid Services (CMS) State Innovation Models (SIM) initiative is a complex policy with heterogeneous state-level implementation.
• Understanding implementation variation is crucial for evaluating policy effectiveness.

Purpose of the Study:

• To present a novel method for characterizing and quantifying variation in the foci and intensity of state implementation of federal policy initiatives.
• To apply this method to the CMS State Innovation Models (SIM) initiative.
• To enable more accurate analysis of policy impacts in natural experiments.

Main Methods:

• Utilized a mixed-methods approach combining quantitative and qualitative data.
• Employed a modified Delphi expert panel process to identify key SIM implementation features impacting health outcomes.
• Researchers scored states based on prioritized features, including stakeholder engagement, behavioral health focus, funding, and payment reforms.

Main Results:

• Identified four critical dimensions of SIM implementation: stakeholder engagement, behavioral health focus, per capita funding, and value-based payment reforms.
• Developed a scoring system that categorized states into high, moderate, and low implementation intensity.
• Demonstrated significant variation in SIM implementation intensity across states.

Conclusions:

• A stakeholder-driven methodology was successfully developed to measure and account for implementation variation in a complex federal policy.
• This approach provides a robust way to characterize state-level implementation differences in initiatives like SIM.
• The developed method can enhance the design and interpretation of natural policy experiments assessing variable policy impacts.