Peer and expert opinion and the reliability of implicit case review

Area of Science:

• Ophthalmology
• Medical Quality Assurance
• Legal Medicine

Background:

• Physician performance evaluation is crucial for patient safety and quality of care.
• Implicit case review is a method used to assess physician performance.
• Assessing agreement among reviewers is key to understanding the reliability of this method.

Observation:

• This study compared interrater and intergroup agreement in judging physician maloccurrence and compliance with standards of care.
• Questionnaires were sent to general ophthalmologists and ophthalmologists with fellowship training.
• Agreement was assessed for judging maloccurrence and compliance with the standard of care.

Findings:

• Overall, 35% of respondents believed physicians committed errors, and 45% believed they did not meet the standard of care.
• Good within-group agreement was found for identifying clinical errors and not meeting standards of care, except for retina specialists.
• Kappa coefficients ranged from 0.55-0.83, but were 0.12 for retina specialists.

Implications:

• Unstructured implicit case review is not a reliable method for determining physician error or measuring compliance with standards of care.
• The process is susceptible to bias, with results potentially varying based on reviewer background and training.
• Implicit case review should be considered a rough screening tool and used cautiously.