Playing smallball: approaches to evaluating pilot health information exchange systems

Health information exchange (HIE) systems are large, multimillion dollar efforts that are implemented despite initial institutional apprehension, with largely unanticipated effects on the clinical workflow, and with a primary goal of establishing a reason to sustain the effort. Traditional evaluation methods that rely on viewing HIE as generalizable tools that can be used to demonstrate impact are challenged by the realities of how HIE systems are created and the numerous systems they impact on the road to improved care. In short, it appears to be unrealistic to develop the definitive "home run" evaluation of this technology. Despite the existence of traditional approaches for large-scale evaluation, a more realistic approach may be a "smallball" model based on established IT implementation phases, with appropriate evaluation dimensions linked to each phase.