Comparing human evaluations of eyewitness statements to a machine learning classifier under pristine and suboptimal...

Area of Science:

• Cognitive psychology
• Machine learning
• Forensic science

Background:

• Machine learning classifiers can distinguish accurate from inaccurate recognition memory decisions.
• There is a growing interest in applying these classifiers to real-world contexts, such as eyewitness confidence statements.

Purpose of the Study:

• To investigate the limitations of verbal machine learning classifiers in applied contexts.
• To compare the performance of verbal classifiers with human evaluators in assessing eyewitness accuracy.
• To examine the impact of confirmatory feedback and lineup information on classifier and human performance.

Main Methods:

• Experiment 1: Assessed the impact of confirmatory feedback on the relationship between identification accuracy and verbal classifier scores.
• Experiment 2: Compared the discriminative value of verbal classifier scores to human evaluators assessing the same verbal confidence statements.
• Manipulated feedback conditions (none vs. confirmatory) and information provided to human evaluators (confidence statement only vs. statement + lineup details).

Main Results:

• Confirmatory feedback weakened the link between identification accuracy and classifier scores, akin to numeric confidence judgments.
• Human evaluators outperformed the classifier when no feedback was given.
• The classifier matched or exceeded human evaluator performance when confirmatory feedback was present.
• Providing lineup information to human evaluators impaired their ability to distinguish correct from filler identifications.

Conclusions:

• Verbal classifiers may be more useful when contextual factors, like lineup presence, hinder human judgment.
• Translating eyewitness statements into classifier scores does not resolve issues stemming from flawed lineup procedures.
• The utility of verbal classifiers is context-dependent and does not universally replace careful lineup administration.