Computerized performance monitoring systems: learning and living with its limitations

Computer technology now allows clinical administrators to collect and analyze large data sets for performance monitoring. Despite the obvious usefulness of this technology, there are limitations. The indices that we can measure are at best proxies that might correlate with good clinical care but can also become dissociated from it in a variety of ways. First, there may not be a relationship throughout the entire continuum between the indicator and what we really value. Second, change in an indicator may not be associated with comparable change in the underlying value. Thirdly, the valence of an indicator can change depending on the context. Fourth, the very act of measuring an indicator can change its valence. Although, from a research perspective there may be technical solutions to these problems, in the real world where clinical care and politics meet, this may not be possible. Indices become reified. Measures become benchmarks and benchmarks quotas. Average is not a statistical phrase but a judgment and below average a term of approbation. To maximize the benefits of computerized monitoring, administrators need to be sensitive to this political dimension.