An investigation of the prevalence of replication research in human factors

The present studies investigated the nature of replication research within the human factors literature.

Many claim that researchers in certain fields do not replicate prior research. This is troubling because replications allow science to self-correct. A successful replication corroborates the original finding, whereas an unsuccessful replication falsifies it. To date, no one has assessed whether this issue affects the field of human factors.

In the first study, eight articles (parent articles) were selected from the 1991 issues of the journal Human Factors. Each article that had referenced one of the eight parent articles between 1991 and September 2006 (child articles) were also retrieved. Two investigators coded and compared each child article against its 1991 parent article to determine whether the child article replicated its parent article. The second study replicated these procedures.

Half or more of the parent articles in Study I and Study 2 (75% and 50%, respectively) were replicated at least once. Furthermore, human factors researchers conducted replications of their own work as well as the work of others. However, many researchers did not state that they replicated previous research.

Replications seem to be common in the human factors literature. However, readers may not realize that a study replicated prior research. Thus, they may incorrectly assess the evidence concerning a given finding.

Human factors professionals should be taught how to identify replications and to be cautious of research that has not been replicated.