The Hawthorne effect in studies of firearm and toolmark examiners

The Hawthorne effect refers to the tendency of individuals to behave differently when they know they are being studied. In the forensic science domain, concerns have been raised about the "strategic examiner," where the forensic examiner uses different decision thresholds depending on whether in a test situation or working on an actual case. The blind testing conducted by the Houston Forensic Science Center ("HFSC") in firearms examination presents a unique opportunity to test the hypothesis that the rate of inconclusive calls differs for discovered vs. undiscovered blind tests of firearm examination. Over 5 years, 529 test item comparisons were filtered into casework at the HFSC. The inconclusive rate for discovered items was 56.4%, while the inconclusive rate for undiscovered test items was 39.3%. Thus, the percentage of inconclusive calls was 43.5% higher among discovered test items than among undiscovered test items. This pattern of results held for bullet comparisons (83% vs. 59%) and cartridge case comparisons (29% vs. 20%) and for both same-source and different-source bullet and cartridge case comparisons. These findings corroborate concerns that examiners behave differently when they know they are being tested and demonstrate the necessity of blind testing if the research goal is to evaluate the performance of forensic examiners conducting casework.