Between- and within-observer agreement for expert judgment of peak flow graphs from a working population

Expert judgment of peak flow-time graphs provides an important tool to detect occupational asthma. This technique has mainly been used in clinics to evaluate reversible airflow obstruction and to assess potential work-related patterns. The reproducibility of this technique in an open working population is unknown. Agreement between and within nine experts was studied using peak flow-time graphs of 49 potato-processing workers. All graphs were classified into four categories by the nine experts, while seven experts read ten graphs at two occasions. Thirty-four graphs (69%) were classified as "no airway obstruction" while four graphs (8%) showed "work-related airway obstruction." There was only slight agreement between the nine experts; mean Cohen's kappa (kappa) was 0.19. Agreement within experts was moderate; mean kappa was 0.47 for judging graphs twice. Our results suggest that in a "healthy" working population, judgment of peak flow graphs is not a favorable method for detection of airway obstruction. If this technique is applied in epidemiological studies, judgment of the graphs should be done by more than one expert.