Forewarning Artificial Intelligence about Cognitive Biases.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Cognitive Science

Background:

• Generative pretrained transformer large language models (LLMs) can exhibit human-like cognitive biases.
• These biases may impact the accuracy and safety of AI-generated medical recommendations.

Purpose of the Study:

• To evaluate if an explicit forewarning can mitigate cognitive biases in an AI model generating medical recommendations.
• To assess the impact of forewarning on the length and content of AI responses.

Main Methods:

• Ten clinically nuanced cases were used to test specific biases with and without a forewarning prompt.
• Responses from the forewarning group were compared to a control group regarding length, discussion of biases, and overall bias reduction.

Main Results:

• Forewarning increased response length by 50% and frequency of bias discussion over 100-fold.
• Despite increased discussion, forewarning only decreased overall bias by 6.9%, with no bias completely eliminated.
• AI models can be prompted to discuss racial and gender bias, but this does not guarantee mitigation of reasoning pitfalls.

Conclusions:

• Explicit forewarning is insufficient to adequately mitigate cognitive biases in large language models for medical recommendations.
• Clinician critical reasoning skills remain essential for interpreting AI-generated medical advice.
• Further research is needed to develop robust methods for bias reduction in medical AI.