Cognitive biases in the peer review of bullet and cartridge case comparison casework: A field study

Forensic judgments and their peer review are often the result of human assessment and are thus subjective and prone to bias. This study examined whether bias affects forensic peer review.

We hypothesized that the probability of disagreement between two forensic examiners about the proposed conclusion would be higher with "blind" peer review (reviewer saw only the first examiner's comparison photos) than with "non-blind" peer review (reviewer also saw the first examiner's interpretation and proposed conclusion). We also hypothesized that examiners with a higher perceived professional status would have a larger effect on the reported conclusion than examiners with a lower status.

We acquired data during a non-blind and a blind peer review procedure in a naturalistic, covert study with eight examiners (3-26 years of experience). We acquired 97 conclusions of bullet and cartridge case comparisons in the blind and 471 in the non-blind peer review procedure.

The odds of disagreement between examiners about the evidential strength of a comparison were approximately five times larger (95%-CI [3.06, 8.50]) in the blind than in the non-blind procedure, with disagreement about 42.3% and 12.5% of the proposed conclusions, respectively. Also, the odds that their proposed conclusion was reported as the final conclusion were approximately 2.5 higher for the higher-status examiners than for lower-status examiners.

Our results support both the hypothesis that bias occurs during non-blind forensic peer review and the hypothesis that higher-status examiners determine the outcome of a discussion more than lower-status examiners. We conclude that blind peer review may reduce the probability of bias and that status effects have an impact on the peer reviewing process.