Improving peer review: alternatives to unstructured judgments by a single reviewer

Peer review, usually involving unstructured judgments by a single peer reviewing medical records, was the backbone of quality management in health care organizations until recent years. Although other approaches to quality management, such as continuous quality improvement, are now being widely adopted, peer review can still play a role in identifying dysfunctional organizational processes and individuals delivering poor care. Given the questionable reliability and validity of peer review as usually practiced, however, a more rigorous and thoughtful approach is needed-with consideration of issues such as the number of peers involved, the nature of their interaction while forming judgments, and the type of assessment instruments they use. MULTIPLE-REVIEWER PROCEDURES: Such procedures include the use of two or more reviewers making their assessments independently, discussion to consensus among reviewers, and committee review.

A number of studies suggest that instruments that carefully guide the reviewer have higher interrater reliability than less structured instruments.

The focus of peer review can be expanded beyond individual practitioner performance to include assessment of the operational environment in which the clinician functions. For example, one might ask whether any changes in hospital policies or procedures or in administrative or computerized support for clinicians might have improved the care received by a patient.

The available data and the known limitations of unstructured judgments by a single reviewer justify serious consideration of multiple reviewer procedures and structured assessment instruments, particularly for reviews that have major consequences for patients and practitioners.