The consequences of AI training on human decision-making

Area of Science:

• Behavioral Science
• Artificial Intelligence Ethics
• Human-Computer Interaction

Background:

• Artificial intelligence (AI) models often learn from human behavior, with the assumption that this training data is unbiased.
• This assumption is critical as AI increasingly influences everyday decisions.

Purpose of the Study:

• To investigate whether humans intentionally modify their behavior when aware their actions are used to train AI.
• To determine if these behavioral changes persist after the AI training phase concludes.

Main Methods:

• Five studies were conducted using the ultimatum game, where participants decided on monetary splits proposed by humans or AI.
• Participants were sometimes informed their choices would train an AI proposer, while others were not.
• Behavioral persistence was assessed by having participants repeat the task without AI training incentives.

Main Results:

• Participants modified their behavior to train AI to make fair proposals, even without direct personal benefit.
• This behavioral shift persisted even after participants were told their responses would no longer be used for AI training, indicating habit formation.
• The findings demonstrate that human behavior as training data can lead to AI perpetuating biases and humans forming non-habitual behaviors.

Conclusions:

• Human behavior used for AI training is not unbiased, as people adapt their actions to shape AI.
• AI algorithms aiming for unbiased human preference representation face challenges due to this behavioral adaptation.
• The study highlights the need to consider the reciprocal influence between humans and AI during the training process.