Explainability does not mitigate the negative impact of incorrect AI advice in a personnel selection task

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence in Human Resources
• Organizational Psychology

Background:

• Artificial intelligence (AI) decision support systems are increasingly used in personnel selection.
• The impact of these AI systems on human decision-making processes remains unclear.
• Understanding human interaction with AI advice is crucial for effective implementation.

Purpose of the Study:

• To investigate how individuals interact with AI-generated advice in personnel selection tasks.
• To examine the influence of advice accuracy, source (human vs. AI), and explainability on decision-making.
• To assess the overreliance on incorrect AI advice and the effect of explainability on mitigating this bias.

Main Methods:

• Five pre-registered experiments involving 1403 students and Human Resource Management (HRM) employees.
• Manipulation of advice accuracy (correct vs. incorrect).
• Manipulation of advice source (human vs. AI) and AI explainability (heatmaps vs. charts).

Main Results:

• Incorrect AI advice significantly impaired decision-making performance due to overreliance.
• Participants failed to dismiss inaccurate advice, indicating a lack of critical evaluation.
• The source of advice and its explainability had limited effects on decision-making outcomes.
• Increased explainability did not reduce overreliance on inaccurate AI predictions.

Conclusions:

• Overreliance on incorrect AI advice is a significant challenge in personnel selection.
• Current explainability methods may not be sufficient to overcome human biases in AI-assisted decision-making.
• There is a critical need for regulatory frameworks and quality standards for AI in HRM to ensure responsible use and mitigate risks.