Reliability of the middle ear examination

Interobserver reliability of physical signs elicited and diagnoses made on middle ear examination were studied. Effect on reliability of variation in observer knowledge, patient age and instrument were defined. Two board-certified, color-seeing pediatric specialists who worked in the same clinic examined consecutively 350 ears with the same otoscope for each case. Over a period of four months, four different otoscopes were used to examine at least 50 ears with each. The following groups of findings were graded independently by the pediatrician according to pre-set criteria: color, landmarks and light reflex. Diagnosis was also recorded. At no time did the observers discuss their findings or diagnoses. Degree of interobserver agreement was expressed by the reliability coefficient (values given below) and the K coefficient. Overall agreement on findings ranged from 0.50 for landmarks to 0.85 for air-fluid level. Overall agreement for diagnoses was 0.72. Specific agreement on physical findings ranged from 0.0 for presence of air-fluid level to 0.85 for absence of air-fluid level. Specific agreement on diagnoses ranged from 0.08 on serous otitis to 0.56 on otitis media and 0.61 on normal. After the age of one year, agreement did not increase with age. Agreement did not change significantly with different instruments or as the study progressed. We concluded that equivalence of observer knowledge strongly increases agreement, while patient age less than one year decreases agreement, and type of instrument does not affect agreement. Clinicians should be trained to recognized middle ear signs and diagnoses according to specified criteria to maximize interobserver agreement.