A system dynamics evaluation model: implementation of health information exchange for public health reporting

To evaluate the complex dynamics involved in implementing electronic health information exchange (HIE) for public health reporting at a state health department, and to identify policy implications to inform similar implementations.

Qualitative data were collected over 8 months from seven experts at New York State Department of Health who implemented web services and protocols for querying, receipt, and validation of electronic data supplied by regional health information organizations. Extensive project documentation was also collected. During group meetings experts described the implementation process and created reference modes and causal diagrams that the evaluation team used to build a preliminary model. System dynamics modeling techniques were applied iteratively to build causal loop diagrams representing the implementation. The diagrams were validated iteratively by individual experts followed by group review online, and through confirmatory review of documents and artifacts.

Three casual loop diagrams captured well-recognized system dynamics: Sliding Goals, Project Rework, and Maturity of Resources. The findings were associated with specific policies that address funding, leadership, ensuring expertise, planning for rework, communication, and timeline management.

This evaluation illustrates the value of a qualitative approach to system dynamics modeling. As a tool for strategic thinking on complicated and intense processes, qualitative models can be produced with fewer resources than a full simulation, yet still provide insights that are timely and relevant.

System dynamics techniques clarified endogenous and exogenous factors at play in a highly complex technology implementation, which may inform other states engaged in implementing HIE supported by federal Health Information Technology for Economic and Clinical Health (HITECH) legislation.