Application of human factors analysis and classification system model to event analysis in radiation oncology

To assess the applicability of the human factors analysis and classification system (HFACS) model for analyzing events in a radiation oncology environment by comparing the HFACS analysis results between novices and experts.

Four novices (resident physicians, 2-4 postgraduate years) volunteered to participate and were asked to independently perform the HFACS analysis on the selected 30 events. The events were divided into the following 2 sets: (1) the description of events was given in detail (eg, the conditions under which the events occurred including information about root causes of error); (2) the description of events was given without detailed information. Each novice's categorization of events into 4 main and 12 sublevels were compared with expert categorization ("gold standard") and used for statistical analysis. Analysis of variance was performed to test the difference in mean agreement with the expert-defined gold standard between the novices across the 4 main levels; and across the 12 sublevels independently.

There were no significant differences in mean agreement with the expert-defined gold standard among the 4 novices (novice versus novice; P > .05) across the 4 main and 12 sublevels, respectively. There was a significant difference in mean agreement with the expert-defined gold standard among the 4 main levels and 12 sublevels (P < .05) across the 4 novices. For the 4 main levels, there was a significant difference in agreement with the expert-defined gold standard for events with detailed information versus events without detailed information provided across the 4 novices. The additional information did not improve mean agreement on the 12 sublevels.

Novices learned to use the HFACS model for higher level analysis (4 main levels) with 1 hour training. Regardless of the amount of detail provided in the event description, the study results indicate a need of formal training for novices to better understand the definition and their interpretation at the 12 sublevel analyses.