Quantifying the strength of firearms comparisons based on error rate studies

Forensic firearms and tool mark examiners compare bullets and cartridge cases to assess whether they originate from the same source or different sources. To communicate their observations, they rely on predefined conclusion scales ranging from Identification to Elimination. However, these terms have not been calibrated against the actual strength of the evidence except indirectly through error rate studies. The present research reanalyzes the findings of firearms and cartridge case comparisons from error rate studies to generate a quantitative measure of the strength of the evidence for each comparison. We use an ordered probit model to summarize the distribution of responses of examiners and aggregate the data for all comparisons to produce a set of likelihood ratios. The likelihood ratios can be as low as less than 10, which does not seem to justify the current articulation scale that may imply a strength of evidence of 10,000 or greater. This suggests that examiners are using language that overstates the strength of the evidence by several orders of magnitude.