Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error...

Area of Science:

• Human-Computer Interaction
• Cognitive Psychology
• Machine Learning Evaluation

Background:

• Machine learning (ML) models are increasingly used to support human decision-making in various tasks.
• Understanding how humans interact with and are affected by ML outputs, especially errors, is crucial for effective deployment.
• Previous research has explored human-AI collaboration, but the specific cognitive impacts of different ML error types require further investigation.

Purpose of the Study:

• To investigate the cognitive effects of correct and incorrect machine learning (ML) outputs on human performance in an object detection task.
• To determine how varying ML model accuracy and error types influence human ability to identify targets and model mistakes.
• To assess the impact of task framing and error importance on human detection of ML errors.

Main Methods:

• Five experiments were conducted using a T and L object detection task (identifying T-shaped targets among L-shaped distractors).
• Participants performed the task with and without ML model outputs (bounding boxes).
• Experiments manipulated ML output accuracy, error proportions (misses vs. false alarms), error importance, and task framing (human vs. model performance focus).

Main Results:

• Model misses were significantly harder for participants to detect than model false alarms.
• Human performance generally improved with higher ML model accuracy, but participants overlooked errors more frequently in highly accurate models.
• Explicit warnings about specific error types had minimal impact on participant performance.

Conclusions:

• Human cognitive processing significantly influences the interpretation and impact of machine learning (ML) model outputs.
• The type of ML error (misses vs. false alarms) has a differential impact on human detection capabilities.
• Acceptable levels of ML performance and error types must consider human cognitive limitations and task context for optimal human-AI collaboration.