The impact of AI errors in a human-in-the-loop process.

Area of Science:

• Human-Computer Interaction
• Public Sector Technology
• Algorithmic Decision-Making

Background:

• Automated decision-making is expanding in the public sector.
• Human oversight is recommended to mitigate algorithmic bias and errors.
• Existing research on human-in-the-loop systems lacks conclusive findings on benefits, risks, and influencing factors.

Purpose of the Study:

• To investigate the impact of human-in-the-loop artificial intelligence (AI) support on decision-making accuracy.
• To examine how the timing of AI feedback (before or after human judgment) affects outcomes.
• To identify specific human-computer interaction aspects influencing automated decision processes.

Main Methods:

• Two experiments simulating automated decision-making in a legal context.
• Participants judged defendants and crimes, with manipulated timing of AI support.
• Independent variable: timing of AI feedback (pre-judgment vs. post-judgment).

Main Results:

• Incorrect AI support negatively impacts human judgment accuracy.
• Pre-judgment AI feedback leads to greater accuracy reduction compared to post-judgment feedback.
• The timing of human-computer interaction with AI is a critical factor in decision quality.

Conclusions:

• Human oversight in automated systems can be detrimental if the AI provides incorrect information.
• The timing of AI feedback is crucial; receiving flawed AI input before judgment significantly reduces accuracy.
• Further research is needed to optimize human-AI collaboration in public sector automated decision-making.