A study of examiner accuracy in cartridge case comparisons. Part 1: Examiner error rates

This report describes a study undertaken to estimate examiner (not laboratory) error rates for false Identifications and false Eliminations when comparing an unknown to a collection of three known cartridge cases. Volunteer active examiners with Association of Firearm and Toolmark Examiners (AFTE) membership or working in laboratories that participate in the Association of Crime Laboratory Directors (ASCLD) were provided with 15 sets of three known and one questioned cartridge cases fired from a collection of 25 new Ruger SR9 handguns. Remington 9-mm Luger (manufacturer designation L9MM3) ammunition was used and comparison sets were made up of cartridge cases fired within 100 cartridges of each other for each gun. Examiners were provided with a background survey, an answer sheet allowing for the AFTE Range of Conclusions, and return shipping materials. In addition to determining whether the known and questioned cartridge cases were fired with the same handgun, examiners were also asked to assess how many of the three knowns in each set were suitable for comparison, providing an estimated rate of how often each firearm used in the study produces useable, quality marks. The participating examiners were provided with both same-source and different-source comparison sets allowing the study to assess both error rates. Responses were received from 218 participating examiners. The overall rate of false Eliminations was estimated as 0.367% from comparisons known to be from the same firearm but reported as Eliminations. The overall rate of false Identifications was estimated as 1.01% from comparisons known to be from different firearms but reported as Identifications. The rates are not uniform across the sample population with a few examiners providing most of the false Identification responses. Rates of poor-quality mark production varied across the 25 sample handguns; those rates were 2.3% ( ± 1.4%). Both false Elimination and false Identification rates are comparable to or lower than the rate of production of poor-quality marks by the firearms used in this study.